Lecture 10 October Minimaxity and least favorable prior sequences
|
|
- Buck Willis
- 6 years ago
- Views:
Transcription
1 STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least favorable prior sequeces I this lecture, we will exted our tools for derivig miimax estimators. Last time, we discovered that miimax estimators ca arise from Bayes estimators uder least favorable priors. However, it turs out that miimax estimators may ot be Bayes estimators. Cosider the followig example, where our old approach fails. Example (Miimax for i.i.d. Normal radom variables with ukow mea ). Let iid X,..., X N (, σ 2 ), with σ 2 kow. Our goal is to estimate uder squared-error loss. For our first guess, pick the atural estimator X. Note that it has costat risk σ2, which suggests miimaxity because we kow that Bayes estimators with costat risk are also miimax estimators. However, X is ot Bayes for ay prior, because uder squared-error loss ubiased estimators are Bayes estimators oly i the degeerate situatios of zero risk (TPE Theorem 4.2.3), ad X is ubiased. Thus, we caot coclude by our previous results (e.g., TPE Corollary 5..5) that X is miimax. We might try to cosider the wider class of estimators δ a,µ0 (X) = ax + ( a) µ 0 for a (0, ) ad µ 0 R, because may of the Bayes estimators we ve ecoutered are covex combiatios of a prior ad a data mea. Note however that the worst case risk for these estimators is ifiite: sup E [ δ (X)] 2 ( ) = sup{a 2 Var X + ( a) 2 ( µ 0 ) 2 } = a2 σ 2 = +. + ( a)2 sup ( µ 0 ) 2 Sice these estimators have poorer worst case risk tha X, they certaily caot be miimax. We could keep tryig to fid Bayes estimators with better worst-case performace tha X, but we would fail: it turs out that X is i fact miimax. To establish this, we will exted our miimax results to the limits of Bayes estimators, rather tha restrictig attetio to Bayes estimators oly. Defiitio (Least Favorable Sequece of Priors). Let {Λ m } be a sequece of priors with miimal average risk r Λm = if δ R (, δ) dλm (). The, {Λ m } is a least favorable sequece of priors if there is a real umber r such that r Λm r < ad r r Λ for ay prior Λ. 0-
2 STATS 300A Lecture 0 October 22 Fall 205 The reaso for studyig the limit of priors is that it may help us establish miimaxity. Sice there eed ot exist a prior Λ such that the associated Bayes estimator has average risk r, this defiitio is less restrictive tha that of a least-favorable prior. We ca prove a aalogue of TPE Theorem 5..4 i this ew settig. Theorem (TPE 5..2). Suppose there is real umber r such that {Λ m } is a sequece of priors with r Λm r <. Let δ be ay estimator such that sup R (, δ) = r. The,. δ is miimax, 2. {Λ m } is least-favorable. Proof.. Let δ be ay other estimator. The, for ay m, sup R (, δ ) R (, δ ) dλ m () r Λm, so that sedig m yields sup which meas that δ is miimax. 2. Let Λ be ay prior, the r Λ = R (, δ Λ ) dλ () which meas that {Λ m } is least favorable. R (, δ ) r = sup R (, δ), R (, δ) dλ () sup R (, δ) = r, Ulike Theorem 5..4, this result does ot guaratee uiqueess, eve if the Bayes estimators δ Λm are uique. This is because the limitig step i the proof of () chages ay strict iequality to ostrict iequality. However, this result allows to check much wider class of estimators, sice to check that the estimator is ideed a miimax estimator we eed to fid oly the sequece of Bayes risks coverget to maximum risk of our cadidate. Example 2 (Miimax for i.i.d. Normal radom variables, cotiued). We ow have the tools to cofirm our suspicio that X is miimax. By Theorem above, it suffices to fid a sequece {Λ m } such that r Λm σ2 =: r. Usig the cojugate prior is a good startig poit, so we let {Λ m } be the cojugate priors {N (0, m 2 )} with variace tedig to, so that Λ m teds to the (improper with π() =, R) uiform prior o R. By TPE Example 4.2.2, the posterior for associated with each Λ m is ( ) X σ X,..., X N 2 +, +. σ 2 m 2 σ 2 m 2 I particular, the posterior variace does ot deped o X,..., X, so Lemma below automatically yields the Bayes risk r Λm = σ 2 + m 2 m σ2 = sup R (, X ). It follows from Theorem that X is miimax ad {Λ m } is least favorable. 0-2
3 STATS 300A Lecture 0 October 22 Fall 205 Lemma (TPE 5..3). If the posterior variace Var Θ X (g (Θ) X = x) is costat i x, the uder squared error loss, r Λ = Var Θ X (g (Θ) X = x). We kow that the posterior mea miimizes Bayes risk, so this result ca be obtaied by pluggig i the posterior mea of g() ito the average risk. 0.2 Miimaxity via submodel restrictio The followig example illustrates the techique of derivig a miimax estimator for a geeral family of models by restrictig attetio to a subset of that family. The idea comes from simple observatio that if the estimator is miimax i submodel ad its risk does t chage whe we go to a larger model the estimator is miimax i this larger class. Example 3 (Miimax for i.i.d. Normal radom variables, ukow mea ad variace). iid Recosider Example i the case that the variace is ukow. That is, let X,..., X N (, σ 2 ), with both ad σ 2 ukow. Note that sup R ( (, σ 2 ), X ) σ 2 = sup,σ 2 σ 2 =, ad i fact, the maximum risk of ay estimator i this settig is ifiite, so the questio of miimaxity is uiterestig. Therefore, we restrict attetio to the family parameterized by Ω = {(, σ 2 ) : R, σ 2 B}, where B is a kow costat. Assume δ is ay other estimator. Calculatig the risk of X withi this family, we fid sup R,σ 2 B R ( (, σ 2 ), X ) = B = sup R ( (, σ 2 ), X ) R,σ 2 =B sup R ( (, σ 2 ), δ ) [submodel miimax] R,σ 2 =B sup R ( (, σ 2 ), δ ), R,σ 2 B where the first iequality follows from the fact that X is miimax for i.i.d. ormals with kow σ 2, ad the secod iequality follows from the fact that we are takig the supremum over a larger set. Hece, we are able to show that X is miimax over Ω by focusig o the case where σ 2 is kow. Notice further that the form of the estimator does ot deped o the upper boud B, though the boud is ecessary for miimaxity to be worth ivestigatig. 0.3 Depedece o the Loss Fuctio I geeral, miimax estimators ca vary depedig o the loss beig cosidered. Below, we provide a example of miimax estimatio uder weighted squared error loss. 0-3
4 STATS 300A Lecture 0 October 22 Fall 205 Example 4 (Miimax for biomial radom variables, weighted squared error loss). Let X Bi (, ) with the loss fuctio L (, d) = (d )2. This is a simple weighted squarederror loss with the weights w() = but it is arguably more realistic tha the usual ( ) ( ) squared error i this situatio because it pealizes errors ear 0 ad more strogly tha errors ear. 2 Note that for ay, R ( ), X = ; that is, the risk is costat i, suggestig X is miimax. We will show that this is ideed the case. We should be careful sice TPE Theorem is oly valid uder the squared-error loss. Sice our loss fuctio is differet, a ubiased estimator ca be Bayes. I this example, this is ideed the case. Recall from TPE Corollary 4..2 that the Bayes estimator associated with the loss L (d, ) = w () (d ) 2 is give by E Θ X[Θw(Θ) X]. E Θ X Ivokig this result, we fid that the [w(θ) X] Bayes estimator has the form δ Λ (X) = E Θ X E Θ X [ [ Θ X] ]. (0.) X Θ( Θ) This is true for arbitrary priors Λ, but to calculate a closed form Bayes estimator, we use a prior cojugate to the biomial likelihood: Θ Λ a,b = Beta (a, b), for some a, b > 0. Suppose we observe X = x. If a + x > ad b + + x >, the substitutig the result of Remark below ito equatio 0. proves that the estimator δ a,b (x) = a + x a + b + 2, miimizes the posterior risk. I particular, the estimator δ, (x) = x miimizes the posterior risk with respect to the uiform prior after observig 0 < x <. If we ca verify that this form remais uchaged whe x {0, }, the the estimator δ, (X) = X is Bayes with costat risk, ad hece miimax. To see that this is the case, ote that the posterior risk uder the prior Λ, after observig X = x ad decidig δ (x) = d is which, i the case X = 0, simplifies to 0 (d ) 2 ( ) Γ (x + + x + ) Γ (x + ) Γ ( x + ) x ( ) x d, 0 (d ) 2 ( ) d. This itegral coverges for d = 0 ad diverges otherwise, so the posterior risk is miimized by choosig δ (0) = 0. Similarly, i the case X =, the posterior risk is miimized by choosig δ () = =. This cofirms that δ, (X) = X miimizes the posterior risk for ay outcome X, ad is ideed Bayes. Sice as we metioed before this estimator has costat risk we ca coclude that X is miimax. Notice that the form of the miimax estimator here depeds o the type of loss beig used: X has costat risk for the type of weighted squared error loss cosidered here. 0-4
5 STATS 300A Lecture 0 October 22 Fall 205 Remark. Recall that the Beta fuctio ca be evaluated as 0 x k ( x) k 2 dx = Γ (k ) Γ (k 2 ) Γ (k + k 2 ), (0.2) wheever k, k 2 > 0. Therefore, if Y Beta (a, b), where a, b > 0, we ca explicitly evaluate the expectatio [ ] E = Y = = [ Γ (a + b) 0 y Γ (a) Γ (b) ya ( y) b Γ (a + b) [ y a ( y) b 2] dy Γ (a) Γ (b) 0 Γ (a + b) Γ (a) Γ (b ) Γ (a) Γ (b) Γ (a + b ) Γ(a + b) Γ(b ) = Γ(a) Γ(a + b ) Γ(b) Γ(a) = a + b b, ] dy where i the secod step we require b > i order to apply the relatio 0.2. A similar argumet yields [ ] (a + b 2) (a + b ) E =, Y ( Y ) (a ) (b ) wheever a >. Combiig these idetities, we have that, wheever a, b >, E [ ] Y [ ] = a E a + b 2. Y ( Y ) 0.4 Radomized Miimax Estimators So far, we have had little occasio to cosider radomized estimators, that is, fuctios δ (X, U) of both the data ad a idepedet source of radomess U Uif (0, ). Radomized estimators played little role i our exploratio of average risk optimality, sice o-radomized estimators of equal or better average risk are always available. However, they tur out to play a role whe we cosider the miimax criterio. Notice that whe workig with covex losses, we ca dispese with radomized estimators, because we ca fid a determiistic estimator with the same or better performace. Ideed, the data X is always sufficiet, so by the Rao-Blackwell theorem, the o-radom estimator δ (X) = E [δ (X, U) X] is o worse tha δ (X, U). However, there are o-covex losses for which o determiistic miimax estimator exists, as the followig example demostrates. 0-5
6 STATS 300A Lecture 0 October 22 Fall δ (i ) δ (i 2 ) δ (i ) legth > α Figure 0.. By choosig α small eough, we ca esure that ay choice of + values for the o-radom estimator δ will leave some 0 a distace at least α away from ay of the δs. Example 5 (Radomized miimax estimator). Let X Bi (, ), where [0, ], ad cosider estimatio of uder the 0- loss, { 0 if d < α L (, d) =. otherwise First cosider a arbitrary o-radom estimator δ. Sice X ca take o oly the + values {0,,..., }, the estimator δ (X) ca take o oly + values, {δ (0), δ (),..., δ ()}. If α <, the we ca always fid 2(+) 0 such that δ (x) 0 α for every x {0,..., }; see Figure 0.. Hece, R ( 0, δ (X)) = is the maximum risk of ay o-radom δ. Cosider istead the estimator δ (X, U) = U, which is completely radom ad idepedet of the data X. The, for ay [0, ], R (, δ ) = E [L (, δ (X, U))] = P ( U α) = P ( α < U < α + ) α<, ad sice α > 0, the maximum risk of δ is smaller tha the maximum risk of ay o-radom δ. Hece, i this settig, there ca be o determiistic miimax estimator. 0-6
Lecture 11 October 27
STATS 300A: Theory of Statistics Fall 205 Lecture October 27 Lecturer: Lester Mackey Scribe: Viswajith Veugopal, Vivek Bagaria, Steve Yadlowsky Warig: These otes may cotai factual ad/or typographic errors..
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationSolutions to Tutorial 3 (Week 4)
The Uiversity of Sydey School of Mathematics ad Statistics Solutios to Tutorial Week 4 MATH2962: Real ad Complex Aalysis Advaced Semester 1, 2017 Web Page: http://www.maths.usyd.edu.au/u/ug/im/math2962/
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More information2 Banach spaces and Hilbert spaces
2 Baach spaces ad Hilbert spaces Tryig to do aalysis i the ratioal umbers is difficult for example cosider the set {x Q : x 2 2}. This set is o-empty ad bouded above but does ot have a least upper boud
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationSeries III. Chapter Alternating Series
Chapter 9 Series III With the exceptio of the Null Sequece Test, all the tests for series covergece ad divergece that we have cosidered so far have dealt oly with series of oegative terms. Series with
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationMath 113, Calculus II Winter 2007 Final Exam Solutions
Math, Calculus II Witer 7 Fial Exam Solutios (5 poits) Use the limit defiitio of the defiite itegral ad the sum formulas to compute x x + dx The check your aswer usig the Evaluatio Theorem Solutio: I this
More informationLecture 15: Learning Theory: Concentration Inequalities
STAT 425: Itroductio to Noparametric Statistics Witer 208 Lecture 5: Learig Theory: Cocetratio Iequalities Istructor: Ye-Chi Che 5. Itroductio Recall that i the lecture o classificatio, we have see that
More informationINFINITE SEQUENCES AND SERIES
INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES I geeral, it is difficult to fid the exact sum of a series. We were able to accomplish this for geometric series ad the series /[(+)]. This is
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More information7 Sequences of real numbers
40 7 Sequeces of real umbers 7. Defiitios ad examples Defiitio 7... A sequece of real umbers is a real fuctio whose domai is the set N of atural umbers. Let s : N R be a sequece. The the values of s are
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationLecture 16: UMVUE: conditioning on sufficient and complete statistics
Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a
More informationBeurling Integers: Part 2
Beurlig Itegers: Part 2 Isomorphisms Devi Platt July 11, 2015 1 Prime Factorizatio Sequeces I the last article we itroduced the Beurlig geeralized itegers, which ca be represeted as a sequece of real umbers
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationAnalytic Continuation
Aalytic Cotiuatio The stadard example of this is give by Example Let h (z) = 1 + z + z 2 + z 3 +... kow to coverge oly for z < 1. I fact h (z) = 1/ (1 z) for such z. Yet H (z) = 1/ (1 z) is defied for
More informationMaximum Likelihood Estimation and Complexity Regularization
ECE90 Sprig 004 Statistical Regularizatio ad Learig Theory Lecture: 4 Maximum Likelihood Estimatio ad Complexity Regularizatio Lecturer: Rob Nowak Scribe: Pam Limpiti Review : Maximum Likelihood Estimatio
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationb i u x i U a i j u x i u x j
M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More informationProblem Set 4 Due Oct, 12
EE226: Radom Processes i Systems Lecturer: Jea C. Walrad Problem Set 4 Due Oct, 12 Fall 06 GSI: Assae Gueye This problem set essetially reviews detectio theory ad hypothesis testig ad some basic otios
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationUniversity of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck!
Uiversity of Colorado Dever Dept. Math. & Stat. Scieces Applied Aalysis Prelimiary Exam 13 Jauary 01, 10:00 am :00 pm Name: The proctor will let you read the followig coditios before the exam begis, ad
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationMATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
More informationCS284A: Representations and Algorithms in Molecular Biology
CS284A: Represetatios ad Algorithms i Molecular Biology Scribe Notes o Lectures 3 & 4: Motif Discovery via Eumeratio & Motif Represetatio Usig Positio Weight Matrix Joshua Gervi Based o presetatios by
More information1 Review and Overview
DRAFT a fial versio will be posted shortly CS229T/STATS231: Statistical Learig Theory Lecturer: Tegyu Ma Lecture #3 Scribe: Migda Qiao October 1, 2013 1 Review ad Overview I the first half of this course,
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationMath 140A Elementary Analysis Homework Questions 3-1
Math 0A Elemetary Aalysis Homework Questios -.9 Limits Theorems for Sequeces Suppose that lim x =, lim y = 7 ad that all y are o-zero. Detarime the followig limits: (a) lim(x + y ) (b) lim y x y Let s
More informationMIDTERM 3 CALCULUS 2. Monday, December 3, :15 PM to 6:45 PM. Name PRACTICE EXAM SOLUTIONS
MIDTERM 3 CALCULUS MATH 300 FALL 08 Moday, December 3, 08 5:5 PM to 6:45 PM Name PRACTICE EXAM S Please aswer all of the questios, ad show your work. You must explai your aswers to get credit. You will
More informationLecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informations = and t = with C ij = A i B j F. (i) Note that cs = M and so ca i µ(a i ) I E (cs) = = c a i µ(a i ) = ci E (s). (ii) Note that s + t = M and so
3 From the otes we see that the parts of Theorem 4. that cocer us are: Let s ad t be two simple o-egative F-measurable fuctios o X, F, µ ad E, F F. The i I E cs ci E s for all c R, ii I E s + t I E s +
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More information6. Uniform distribution mod 1
6. Uiform distributio mod 1 6.1 Uiform distributio ad Weyl s criterio Let x be a seuece of real umbers. We may decompose x as the sum of its iteger part [x ] = sup{m Z m x } (i.e. the largest iteger which
More informationMA131 - Analysis 1. Workbook 3 Sequences II
MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................
More informationMA131 - Analysis 1. Workbook 9 Series III
MA3 - Aalysis Workbook 9 Series III Autum 004 Cotets 4.4 Series with Positive ad Negative Terms.............. 4.5 Alteratig Series.......................... 4.6 Geeral Series.............................
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More informationLECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if
LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationCHAPTER 10 INFINITE SEQUENCES AND SERIES
CHAPTER 10 INFINITE SEQUENCES AND SERIES 10.1 Sequeces 10.2 Ifiite Series 10.3 The Itegral Tests 10.4 Compariso Tests 10.5 The Ratio ad Root Tests 10.6 Alteratig Series: Absolute ad Coditioal Covergece
More informationChapter 4. Fourier Series
Chapter 4. Fourier Series At this poit we are ready to ow cosider the caoical equatios. Cosider, for eample the heat equatio u t = u, < (4.) subject to u(, ) = si, u(, t) = u(, t) =. (4.) Here,
More information2.1. The Algebraic and Order Properties of R Definition. A binary operation on a set F is a function B : F F! F.
CHAPTER 2 The Real Numbers 2.. The Algebraic ad Order Properties of R Defiitio. A biary operatio o a set F is a fuctio B : F F! F. For the biary operatios of + ad, we replace B(a, b) by a + b ad a b, respectively.
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationMath 299 Supplement: Real Analysis Nov 2013
Math 299 Supplemet: Real Aalysis Nov 203 Algebra Axioms. I Real Aalysis, we work withi the axiomatic system of real umbers: the set R alog with the additio ad multiplicatio operatios +,, ad the iequality
More informationMath 341 Lecture #31 6.5: Power Series
Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series
More informationComplex Analysis Spring 2001 Homework I Solution
Complex Aalysis Sprig 2001 Homework I Solutio 1. Coway, Chapter 1, sectio 3, problem 3. Describe the set of poits satisfyig the equatio z a z + a = 2c, where c > 0 ad a R. To begi, we see from the triagle
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More information6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.
6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio
More informationLECTURES 5 AND 6: CONVERGENCE TESTS
LECTURES 5 AND 6: CONVERGENCE TESTS I the previous lecture we saw a very basic test for covergece - the compariso test. Here we develop slightly more sophisticated machiery for determiig the covergece
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationSequences I. Chapter Introduction
Chapter 2 Sequeces I 2. Itroductio A sequece is a list of umbers i a defiite order so that we kow which umber is i the first place, which umber is i the secod place ad, for ay atural umber, we kow which
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationFeedback in Iterative Algorithms
Feedback i Iterative Algorithms Charles Byre (Charles Byre@uml.edu), Departmet of Mathematical Scieces, Uiversity of Massachusetts Lowell, Lowell, MA 01854 October 17, 2005 Abstract Whe the oegative system
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationTHE INTEGRAL TEST AND ESTIMATES OF SUMS
THE INTEGRAL TEST AND ESTIMATES OF SUMS. Itroductio Determiig the exact sum of a series is i geeral ot a easy task. I the case of the geometric series ad the telescoig series it was ossible to fid a simle
More informationREAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS
REAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS 18th Feb, 016 Defiitio (Lipschitz fuctio). A fuctio f : R R is said to be Lipschitz if there exists a positive real umber c such that for ay x, y i the domai
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationLecture 9: September 19
36-700: Probability ad Mathematical Statistics I Fall 206 Lecturer: Siva Balakrisha Lecture 9: September 9 9. Review ad Outlie Last class we discussed: Statistical estimatio broadly Pot estimatio Bias-Variace
More informationSequences. Notation. Convergence of a Sequence
Sequeces A sequece is essetially just a list. Defiitio (Sequece of Real Numbers). A sequece of real umbers is a fuctio Z (, ) R for some real umber. Do t let the descriptio of the domai cofuse you; it
More informationMathematical Methods for Physics and Engineering
Mathematical Methods for Physics ad Egieerig Lecture otes Sergei V. Shabaov Departmet of Mathematics, Uiversity of Florida, Gaiesville, FL 326 USA CHAPTER The theory of covergece. Numerical sequeces..
More information1 Review and Overview
CS9T/STATS3: Statistical Learig Theory Lecturer: Tegyu Ma Lecture #6 Scribe: Jay Whag ad Patrick Cho October 0, 08 Review ad Overview Recall i the last lecture that for ay family of scalar fuctios F, we
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More information5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY
IA Probability Let Term 5 INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY 51 Iequalities Suppose that X 0 is a radom variable takig o-egative values ad that c > 0 is a costat The P X c E X, c is
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationREGRESSION WITH QUADRATIC LOSS
REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More information