Entropy and Ergodic Theory Notes 10: Large Deviations I
|
|
- Kristian Harmon
- 6 years ago
- Views:
Transcription
1 Entropy nd Ergodic Theory Notes 10: Lrge Devitions I 1 A chnge of convention This is our first lecture on pplictions of entropy in probbility theory. In probbility theory, the convention is tht ll logrithms re nturl unless stted otherwise. So, strting in this lecture, we dopt this convention for entropy: in the formul H(p) = A p() log p(), the logrithm is to bse e. 2 A vrint of the bsic counting problem; divergence This lecture strts with probbilistic vrint of our bsic counting problem from the strt of the course. Let p Prob(A), nd suppose tht X = (X 1,..., X n ) p n. Suppose lso tht x A n is fixed string. We hve seen tht if p x = p, then the probbility P(X = x) is e H(p)n (previously it ws written 2 H(p)n, since tht ws before we switched from log 2 to nturl logrithms). This fct ws the bsis of our originl upper bound on the number of strings of type p. But now suppose tht p x is some other element q Prob(A). Then reltive of the erlier clcultion gives P(X = x) = p() N( x) = p() nq(). 1
2 If there exists A for which p() = 0 but q() 0, then this probbility is zero. Otherwise, we my write it s [( ) ] exp q() log p() n. (1) In fct, we cn write it this wy in both cses, provided we dopt the nturl conventions log 0 := nd exp( ) := 0. Therefore [( ) ] P(p X = q) = P(X T n (q)) = T n (q) exp q() log p() n. In cse denom(q) divides n, our originl estimtes for T n (q) give [( exp q() log q() + ) ] q() log p() n o(n) [ ( = exp q() log q() ) ] n o(n) p() P(p X = q) [ ( exp q() log q() ) ] n. (2) p() Tht is, we hve estimted the probbility tht rndom string drwn i.i.d. from p hs type equl to some other distribution q. The min negtive term in the exponent hs coefficient q() log q() p() = p() q() q() log p() p(). This is clled the divergence (or Kullbck Liebler divergence, informtion divergence, or reltive entropy) of q with respect to p. We denoted it by D(q p). If there exists A such tht p() = 0 but q() 0, then D(q p) = + by definition. Exmple. If p is the uniform distribution on Prob(A), then p(a) = 1/ A for every A, nd D(q p) = log A H(q) q Prob(A). (3) 2
3 In order to hndle the cse of infinite divergence succinctly, recll the following notion nd nottion from mesure theory. If (X, B) is mesurble spce nd µ nd ν re two probbility mesures on it, then ν is bsolutely continuous with respect to µ, denoted ν µ, if ν(a) = 0 whenever µ(a) = 0 for A B. If X is countble nd B is the power set of X, then it is enough to check this for ll singletons A. Lemm 1. For fixed p, the quntity D(q p) is continuous nd non-negtive function on the set {q Prob(A) : q p}. It is zero if nd only if q = p. The quntity D(q p) is strictly convex s function of the pir (q, p) Prob(A) Prob(A) (this is stronger thn being strictly convex in ech rgument seprtely). Proof. Continuity is cler. To prove non-negtivity, recll tht the function η(t) := t log t is strictly convex on (0, ). Since p is probbility distribution, Jensen s inequlity gives D(q p) = ( q() ) ( p()η η p() q() ) = η(1) = 0. p() p() Equlity occurs if nd only if q()/p() is the sme vlue for every. Since q nd p re both probbility distributions, this occurs only if q = p. We leve joint strict convexity s n exercise for the reder (it requires nother, slightly less obvious ppliction of Jensen s inequlity). nd In terms of divergence, we cn re-write the clcultions (1) nd (2) s P(X = x) = p n [H(px)+D(px p)]n (x) = e e D(q p)n o(n) P(p X = q) = e D(q p)n in cse denom(q) = n. One importnt exmple of divergence hs lredy been introduced by nother nme. 3
4 Exmple. Let p Prob(A B) hve mrginls p A nd p B. Let (X, Y ) p. In Lecture 2 we derived the formul I(X ; Y ) =,b p(, b) log p(, b) p A ()p B (b) (except with log 2 in plce of log, similrly to the rest of this lecture). Thus, the mutul informtion between X nd Y is equl to D(p p A p B ). It my be seen s compring the true joint distribution p to the distribution of independent RVs tht hve the sme individul distributions s X nd Y. 3 Snov s theorem The clcultion in (2) cn be seen s quntittive strengthening of the LLN for types. If q p, then D(q p) > 0, nd so we hve shown tht for X p n the event {p X = q} is not only unlikely, but exponentilly unlikely, with exponent given by D(q p). By some simple estimtes nd summing over possible types, this cn be turned into n estimte for the following more generl question: If X p n nd U Prob(A) is such tht {p X U} is unlikely, then just how unlikely is it? To lighten nottion, let us now write T n (U) := {x A n : p x U} = p U T n (p) for ny n N nd U Prob(A). Theorem 2 (Snov s theorem). Every U Prob(A) stisfies ( exp [ ) inf D(q p)] n + o(n) q U P(p X U) ( exp [ inf q U D(q p)] n + o(n) where U is the interior of U for the usul topology of Prob(A). ), 4
5 Corollry 3. If U Prob(A) hs the property tht U U (tht is, the interior of U is dense in U), then ( P(p X U) = exp [ ) inf D(q p)] n + o(n). q U Proof. Upper bound. This is nother exercise in summing over typicl sets. Let d := inf q U D(q p). Then the upper bound (2) gives P(p X U) = q U P(p X T n (q)) q U, denom(q) n D(q p)n e {q U : denom(q) = n} e dn (n + 1) A e dn = e dn+o(n). Lower bound. We need the following fct: For ll sufficiently lrge n is holds tht, for ny p Prob(A), there exists q Prob(A) with denomintor n such tht p q 2 A /n. This ws lredy used in Lecture 2, where we referred to it s (D). Now let d := inf q U D(q p). Let ε > 0, nd let V U be nonempty open subset such tht ny q V stisfies D(q p) < d + ε. Since V is open, fct (D) promises tht, provided n is sufficiently lrge, there exists q V of denomintor n. For this q, we my now pply the lower bound from (2) to obtin P(p X U) P(p X = q) = e D(q p)n o(n) e (d +ε)n o(n). Since ε ws rbitrry, this ctully holds for some slowly decresing sequence ε n 0 in plce of ε. This completes the proof. Using Lemm 1, we immeditely deduce the following. Corollry 4. If p Prob(A) nd U Prob(A) stisfy p U, nd if X p n, then P(p X U) decys exponentilly fst s n. Exmple. Let p Prob(A B) hve mrginls p A nd p B. Let (X, Y) p n. Then we hve seen tht I(X i ; Y i ) = D(p p A p B ) for ech i (except with log 2 in plce of log, similrly to the rest of this lecture). 5
6 Now suppose we drw rndom strings X p n nd Y p n independently of ech other. The LLN for types gives tht X nd Y re pproximtely typicl for p A nd p B with high probbility. But if p p A p B, then (X, Y ) is unlikely to be pproximtely typicl for p. Snov s theorem lets us estimte this smll probbility: for δ > 0, tht theorem gives P((X, Y ) T n,δ (p)) = e D(p p A p B )n+ (δ)n+o(n) = e I(X i ; Y i )n+ (δ)n+o(n). This cn lso be obtined from our erlier description of the typicl set T n,δ (p) in terms of conditionl entropy. 4 The conditionl limit theorem Snov s Theorem cn be used to nswer the following vrint of the previous question: Suppose tht X p n nd tht E Prob(A) is such tht {p X E} is unlikely. Conditionlly on the unlikely occurrence {p X E}, wht is the most likely behviour of p X? (Put nother wy, wht the lest unlikely wy for this unlikely event to occur?) We cn nswer this rther completely in cse E is closed, convex, nd hs nonempty interior (in Prob(A)). In tht cse, if p X does lnd in the unlikely set E, then it turns out tht it is overwhelmingly likely to be close to prticulr distinguished element of E. Lemm 5. If p Prob(A) nd if E Prob(A) is closed nd convex, then the function D( p) hs unique minimizer on E. Proof. This follows t once from the continuity nd strict convexity of D( p). Theorem 6. Let E Prob(A) be closed, convex, nd hve nonempty interior, nd suppose tht p Prob(A) \ E. Let q be the unique minimizer of D( p) in E, nd ssume tht D(q p) <. Then, for ny ε > 0, we hve P ( p X q < ε px E ) 1 s n, nd the convergence is exponentil in n. 6
7 Remrk. The ssumption tht D(q p) < merely excludes degenerte cses: if this does not hold, then the event {p X E} simply hs zero probbility. Proof. Let d := inf q E D(q p) = D(q p). Let us prtition E s (E B ε (q )) (E \ B ε (q )), where B ε (q ) is the open ε-bll round q in totl vrition. Assume ε is smll enough tht both prts re nonempty (otherwise the desired probbility is lwys equl to 1). Since q is the unique minimizer on E, nd since E \ B ε (q ) is closed nd hence compct, we hve d := min D(q p) > d. q E\B ε(q ) Now pply Snov s theorem to both of the sets E nd E \ B ε (q ): P(p X E) = e dn+o(n) nd P(p X E \ B ε (q )) e d n+o(n). The first of these is true symptotic becuse E E, nd so we my ctully pply Corollry 3. The second estimte here is just the upper bound from Theorem 2. Combining these, we obtin P ( p X q ε px E ) = P(p X E \ B ε (q )) P(p X E) = e (d d)n+o(n). This tends to 0 exponentilly fst s n. Corollry 7 (The conditionl limit theorem). Under the conditions of the previous theorem, we hve P ( X 1 = px E ) q () s n for every A. Proof. The lw of X nd the event {p X E} re both invrint under permuting the entries of the rndom string X = (X 1,..., X n ). Therefore the vlue P ( X i = px E ) is the sme for every i [n], nd so it equls 1 n n i=1 P ( X i = px E ) ( 1 = E n n i=1 1 {Xi =} 7 ) p X E = E ( p X () px E ).
8 By Theorem 2, under the conditioned mesure P( p X E), the empiricl distribution stys within ny ε of q with high probbility s n, nd so the bove expecttion lso stys within ny ε of q () s n tht is, it converges to q (). With similr proof nd just little extr work, one obtins the following strengthening. Corollry 8 (Multiple conditionl limit theorem). Under the sme conditions s bove, nd for ny fixed m N, we hve P ( X 1 = 1, X 2 = 2,..., X m = m px E ) q ( 1 )q ( 2 ) q ( m ) s n for every 1,..., m A. 5 Notes nd remrks Our bsic source for this lecture is [CT06]: For divergence nd its properties, Sections (lso, see the end of Section 2.5 for conditionl version of divergence); For the probbility of type clss: Theorem ; Snov s theorem: Theorem ; The conditionl limit theorem: Theorem See the rest of [CT06, Section 11.6] for finer nlysis of the reltion between the quntities D(q p) nd q p. See the remining sections of [CT06, Chpter 11] for some vluble pplictions to sttistics. Snov s theorem nd the conditionl limit theorem re just the strt of the lrge subfield of probbility clled lrge devitions theory. Our next lecture is lso given to results from this field. Severl good sources re vilble to tke the reder further in this direction: The book [DZ10] is dedicted to lrge devitions theory, nd introduces mny spects of the subject t quite gentle pce. See, for instnce, [DZ10, Theorem 3.3.3] for the generliztion of the conditionl limit theorem to non-convex sets E. 8
9 The lecture notes [Vr] go rther fster, but re still cler nd introduce severl interesting dvnced topics. The monogrph [Ell06] lso goes quite gently nd emphsized the connection to sttisticl physics. I will im to mke this connection bit lter in the course. The clssic rticle [Ln73] hs similr point of view, but requires little more mthemticl bckground. Section A4 of tht rticle is on lrge devitions. Severl refinements re known to the conditionl limit theorem. The ppers [Csi75, Csi84, DZ96] cover severl of these, nd explin the subject very clerly. See [Kul97] for much more on the ppliction of informtion theoretic quntities in sttistics. References [Csi75] I. Csiszár. I-divergence geometry of probbility distributions nd minimiztion problems. Ann. Probbility, 3: , [Csi84] Imre Csiszár. Snov property, generlized I-projection nd conditionl limit theorem. Ann. Probb., 12(3): , [CT06] Thoms M. Cover nd Joy A. Thoms. Elements of informtion theory. Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, second edition, [DZ96] A. Dembo nd O. Zeitouni. Refinements of the Gibbs conditioning principle. Probb. Theory Relted Fields, 104(1):1 14, [DZ10] Amir Dembo nd Ofer Zeitouni. Lrge devitions techniques nd pplictions, volume 38 of Stochstic Modelling nd Applied Probbility. Springer-Verlg, Berlin, Corrected reprint of the second (1998) edition. [Ell06] Richrd S. Ellis. Entropy, lrge devitions, nd sttisticl mechnics. Clssics in Mthemtics. Springer-Verlg, Berlin, Reprint of the 1985 originl. 9
10 [Kul97] Solomon Kullbck. Informtion theory nd sttistics. Dover Publictions, Inc., Mineol, NY, Reprint of the second (1968) edition. [Ln73] O.E. Lnford III. Entropy nd equilibrium sttes in clssicl sttisticl mechnics. In conference proceedings on Sttisticl Mechnics nd Mthemticl Problems. Springer, [Vr] S. R. S. Vrdhn. Lrge devitions nd pplictions. Number 46 in CBMS-NSF Regionl Conference Series in Applied Mthemtics. Society for Industril nd Applied Mthemtics (SIAM). TIM AUSTIN Emil: URL: mth.ucl.edu/ tim 10
p-adic Egyptian Fractions
p-adic Egyptin Frctions Contents 1 Introduction 1 2 Trditionl Egyptin Frctions nd Greedy Algorithm 2 3 Set-up 3 4 p-greedy Algorithm 5 5 p-egyptin Trditionl 10 6 Conclusion 1 Introduction An Egyptin frction
More informationW. We shall do so one by one, starting with I 1, and we shall do it greedily, trying
Vitli covers 1 Definition. A Vitli cover of set E R is set V of closed intervls with positive length so tht, for every δ > 0 nd every x E, there is some I V with λ(i ) < δ nd x I. 2 Lemm (Vitli covering)
More informationThe Regulated and Riemann Integrals
Chpter 1 The Regulted nd Riemnn Integrls 1.1 Introduction We will consider severl different pproches to defining the definite integrl f(x) dx of function f(x). These definitions will ll ssign the sme vlue
More informationAdvanced Calculus: MATH 410 Notes on Integrals and Integrability Professor David Levermore 17 October 2004
Advnced Clculus: MATH 410 Notes on Integrls nd Integrbility Professor Dvid Levermore 17 October 2004 1. Definite Integrls In this section we revisit the definite integrl tht you were introduced to when
More informationDiscrete Mathematics and Probability Theory Spring 2013 Anant Sahai Lecture 17
EECS 70 Discrete Mthemtics nd Proility Theory Spring 2013 Annt Shi Lecture 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion,
More informationDiscrete Mathematics and Probability Theory Summer 2014 James Cook Note 17
CS 70 Discrete Mthemtics nd Proility Theory Summer 2014 Jmes Cook Note 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion, y tking
More information8 Laplace s Method and Local Limit Theorems
8 Lplce s Method nd Locl Limit Theorems 8. Fourier Anlysis in Higher DImensions Most of the theorems of Fourier nlysis tht we hve proved hve nturl generliztions to higher dimensions, nd these cn be proved
More informationTheoretical foundations of Gaussian quadrature
Theoreticl foundtions of Gussin qudrture 1 Inner product vector spce Definition 1. A vector spce (or liner spce) is set V = {u, v, w,...} in which the following two opertions re defined: (A) Addition of
More informationLecture 1. Functional series. Pointwise and uniform convergence.
1 Introduction. Lecture 1. Functionl series. Pointwise nd uniform convergence. In this course we study mongst other things Fourier series. The Fourier series for periodic function f(x) with period 2π is
More informationRiemann is the Mann! (But Lebesgue may besgue to differ.)
Riemnn is the Mnn! (But Lebesgue my besgue to differ.) Leo Livshits My 2, 2008 1 For finite intervls in R We hve seen in clss tht every continuous function f : [, b] R hs the property tht for every ɛ >
More informationRiemann Sums and Riemann Integrals
Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 203 Outline Riemnn Sums Riemnn Integrls Properties Abstrct
More informationMath 1B, lecture 4: Error bounds for numerical methods
Mth B, lecture 4: Error bounds for numericl methods Nthn Pflueger 4 September 0 Introduction The five numericl methods descried in the previous lecture ll operte by the sme principle: they pproximte the
More informationRiemann Sums and Riemann Integrals
Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 2013 Outline 1 Riemnn Sums 2 Riemnn Integrls 3 Properties
More informationReview of Riemann Integral
1 Review of Riemnn Integrl In this chpter we review the definition of Riemnn integrl of bounded function f : [, b] R, nd point out its limittions so s to be convinced of the necessity of more generl integrl.
More informationarxiv:math/ v2 [math.ho] 16 Dec 2003
rxiv:mth/0312293v2 [mth.ho] 16 Dec 2003 Clssicl Lebesgue Integrtion Theorems for the Riemnn Integrl Josh Isrlowitz 244 Ridge Rd. Rutherford, NJ 07070 jbi2@njit.edu Februry 1, 2008 Abstrct In this pper,
More informationThe First Fundamental Theorem of Calculus. If f(x) is continuous on [a, b] and F (x) is any antiderivative. f(x) dx = F (b) F (a).
The Fundmentl Theorems of Clculus Mth 4, Section 0, Spring 009 We now know enough bout definite integrls to give precise formultions of the Fundmentl Theorems of Clculus. We will lso look t some bsic emples
More informationChapter 4 Contravariance, Covariance, and Spacetime Diagrams
Chpter 4 Contrvrince, Covrince, nd Spcetime Digrms 4. The Components of Vector in Skewed Coordintes We hve seen in Chpter 3; figure 3.9, tht in order to show inertil motion tht is consistent with the Lorentz
More informationLecture notes. Fundamental inequalities: techniques and applications
Lecture notes Fundmentl inequlities: techniques nd pplictions Mnh Hong Duong Mthemtics Institute, University of Wrwick Emil: m.h.duong@wrwick.c.uk Februry 8, 207 2 Abstrct Inequlities re ubiquitous in
More informationEuler, Ioachimescu and the trapezium rule. G.J.O. Jameson (Math. Gazette 96 (2012), )
Euler, Iochimescu nd the trpezium rule G.J.O. Jmeson (Mth. Gzette 96 (0), 36 4) The following results were estblished in recent Gzette rticle [, Theorems, 3, 4]. Given > 0 nd 0 < s
More informationUnit #9 : Definite Integral Properties; Fundamental Theorem of Calculus
Unit #9 : Definite Integrl Properties; Fundmentl Theorem of Clculus Gols: Identify properties of definite integrls Define odd nd even functions, nd reltionship to integrl vlues Introduce the Fundmentl
More informationContinuous Random Variables
STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 217 Néhémy Lim Continuous Rndom Vribles Nottion. The indictor function of set S is rel-vlued function defined by : { 1 if x S 1 S (x) if x S Suppose tht
More informationReversals of Signal-Posterior Monotonicity for Any Bounded Prior
Reversls of Signl-Posterior Monotonicity for Any Bounded Prior Christopher P. Chmbers Pul J. Hely Abstrct Pul Milgrom (The Bell Journl of Economics, 12(2): 380 391) showed tht if the strict monotone likelihood
More informationHandout: Natural deduction for first order logic
MATH 457 Introduction to Mthemticl Logic Spring 2016 Dr Json Rute Hndout: Nturl deduction for first order logic We will extend our nturl deduction rules for sententil logic to first order logic These notes
More informationProperties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives
Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums - 1 Riemnn
More informationReview of basic calculus
Review of bsic clculus This brief review reclls some of the most importnt concepts, definitions, nd theorems from bsic clculus. It is not intended to tech bsic clculus from scrtch. If ny of the items below
More informationA REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H. Thomas Shores Department of Mathematics University of Nebraska Spring 2007
A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H Thoms Shores Deprtment of Mthemtics University of Nebrsk Spring 2007 Contents Rtes of Chnge nd Derivtives 1 Dierentils 4 Are nd Integrls 5 Multivrite Clculus
More informationRecitation 3: More Applications of the Derivative
Mth 1c TA: Pdric Brtlett Recittion 3: More Applictions of the Derivtive Week 3 Cltech 2012 1 Rndom Question Question 1 A grph consists of the following: A set V of vertices. A set E of edges where ech
More informationUNIFORM CONVERGENCE. Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3
UNIFORM CONVERGENCE Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3 Suppose f n : Ω R or f n : Ω C is sequence of rel or complex functions, nd f n f s n in some sense. Furthermore,
More informationSUMMER KNOWHOW STUDY AND LEARNING CENTRE
SUMMER KNOWHOW STUDY AND LEARNING CENTRE Indices & Logrithms 2 Contents Indices.2 Frctionl Indices.4 Logrithms 6 Exponentil equtions. Simplifying Surds 13 Opertions on Surds..16 Scientific Nottion..18
More informationMath 61CM - Solutions to homework 9
Mth 61CM - Solutions to homework 9 Cédric De Groote November 30 th, 2018 Problem 1: Recll tht the left limit of function f t point c is defined s follows: lim f(x) = l x c if for ny > 0 there exists δ
More information20 MATHEMATICS POLYNOMIALS
0 MATHEMATICS POLYNOMIALS.1 Introduction In Clss IX, you hve studied polynomils in one vrible nd their degrees. Recll tht if p(x) is polynomil in x, the highest power of x in p(x) is clled the degree of
More informationImproper Integrals, and Differential Equations
Improper Integrls, nd Differentil Equtions October 22, 204 5.3 Improper Integrls Previously, we discussed how integrls correspond to res. More specificlly, we sid tht for function f(x), the region creted
More informationMath 360: A primitive integral and elementary functions
Mth 360: A primitive integrl nd elementry functions D. DeTurck University of Pennsylvni October 16, 2017 D. DeTurck Mth 360 001 2017C: Integrl/functions 1 / 32 Setup for the integrl prtitions Definition:
More informationECO 317 Economics of Uncertainty Fall Term 2007 Notes for lectures 4. Stochastic Dominance
Generl structure ECO 37 Economics of Uncertinty Fll Term 007 Notes for lectures 4. Stochstic Dominnce Here we suppose tht the consequences re welth mounts denoted by W, which cn tke on ny vlue between
More informationPolynomial Approximations for the Natural Logarithm and Arctangent Functions. Math 230
Polynomil Approimtions for the Nturl Logrithm nd Arctngent Functions Mth 23 You recll from first semester clculus how one cn use the derivtive to find n eqution for the tngent line to function t given
More information1 The Lagrange interpolation formula
Notes on Qudrture 1 The Lgrnge interpoltion formul We briefly recll the Lgrnge interpoltion formul. The strting point is collection of N + 1 rel points (x 0, y 0 ), (x 1, y 1 ),..., (x N, y N ), with x
More informationDuality # Second iteration for HW problem. Recall our LP example problem we have been working on, in equality form, is given below.
Dulity #. Second itertion for HW problem Recll our LP emple problem we hve been working on, in equlity form, is given below.,,,, 8 m F which, when written in slightly different form, is 8 F Recll tht we
More informationDISCRETE MATHEMATICS HOMEWORK 3 SOLUTIONS
DISCRETE MATHEMATICS 21228 HOMEWORK 3 SOLUTIONS JC Due in clss Wednesdy September 17. You my collborte but must write up your solutions by yourself. Lte homework will not be ccepted. Homework must either
More informationRIEMANN-LIOUVILLE AND CAPUTO FRACTIONAL APPROXIMATION OF CSISZAR S f DIVERGENCE
SARAJEVO JOURNAL OF MATHEMATICS Vol.5 (17 (2009, 3 12 RIEMANN-LIOUVILLE AND CAPUTO FRACTIONAL APPROIMATION OF CSISZAR S f DIVERGENCE GEORGE A. ANASTASSIOU Abstrct. Here re estblished vrious tight probbilistic
More informationAdvanced Calculus: MATH 410 Uniform Convergence of Functions Professor David Levermore 11 December 2015
Advnced Clculus: MATH 410 Uniform Convergence of Functions Professor Dvid Levermore 11 December 2015 12. Sequences of Functions We now explore two notions of wht it mens for sequence of functions {f n
More informationAppendix to Notes 8 (a)
Appendix to Notes 8 () 13 Comprison of the Riemnn nd Lebesgue integrls. Recll Let f : [, b] R be bounded. Let D be prtition of [, b] such tht Let D = { = x 0 < x 1
More information1 Probability Density Functions
Lis Yn CS 9 Continuous Distributions Lecture Notes #9 July 6, 28 Bsed on chpter by Chris Piech So fr, ll rndom vribles we hve seen hve been discrete. In ll the cses we hve seen in CS 9, this ment tht our
More informationQuadratic Forms. Quadratic Forms
Qudrtic Forms Recll the Simon & Blume excerpt from n erlier lecture which sid tht the min tsk of clculus is to pproximte nonliner functions with liner functions. It s ctully more ccurte to sy tht we pproximte
More informationCHAPTER 4 MULTIPLE INTEGRALS
CHAPTE 4 MULTIPLE INTEGAL The objects of this chpter re five-fold. They re: (1 Discuss when sclr-vlued functions f cn be integrted over closed rectngulr boxes in n ; simply put, f is integrble over iff
More informationChapter 5 : Continuous Random Variables
STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 216 Néhémy Lim Chpter 5 : Continuous Rndom Vribles Nottions. N {, 1, 2,...}, set of nturl numbers (i.e. ll nonnegtive integers); N {1, 2,...}, set of ll
More informationAn approximation to the arithmetic-geometric mean. G.J.O. Jameson, Math. Gazette 98 (2014), 85 95
An pproximtion to the rithmetic-geometric men G.J.O. Jmeson, Mth. Gzette 98 (4), 85 95 Given positive numbers > b, consider the itertion given by =, b = b nd n+ = ( n + b n ), b n+ = ( n b n ) /. At ech
More information1B40 Practical Skills
B40 Prcticl Skills Comining uncertinties from severl quntities error propgtion We usully encounter situtions where the result of n experiment is given in terms of two (or more) quntities. We then need
More informationFUNDAMENTALS OF REAL ANALYSIS by. III.1. Measurable functions. f 1 (
FUNDAMNTALS OF RAL ANALYSIS by Doğn Çömez III. MASURABL FUNCTIONS AND LBSGU INTGRAL III.. Mesurble functions Hving the Lebesgue mesure define, in this chpter, we will identify the collection of functions
More informationChapter 0. What is the Lebesgue integral about?
Chpter 0. Wht is the Lebesgue integrl bout? The pln is to hve tutoril sheet ech week, most often on Fridy, (to be done during the clss) where you will try to get used to the ides introduced in the previous
More information1 Online Learning and Regret Minimization
2.997 Decision-Mking in Lrge-Scle Systems My 10 MIT, Spring 2004 Hndout #29 Lecture Note 24 1 Online Lerning nd Regret Minimiztion In this lecture, we consider the problem of sequentil decision mking in
More informationLecture 1: Introduction to integration theory and bounded variation
Lecture 1: Introduction to integrtion theory nd bounded vrition Wht is this course bout? Integrtion theory. The first question you might hve is why there is nything you need to lern bout integrtion. You
More informationRiemann Integrals and the Fundamental Theorem of Calculus
Riemnn Integrls nd the Fundmentl Theorem of Clculus Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University September 16, 2013 Outline Grphing Riemnn Sums
More informationReview of Calculus, cont d
Jim Lmbers MAT 460 Fll Semester 2009-10 Lecture 3 Notes These notes correspond to Section 1.1 in the text. Review of Clculus, cont d Riemnn Sums nd the Definite Integrl There re mny cses in which some
More informationThe steps of the hypothesis test
ttisticl Methods I (EXT 7005) Pge 78 Mosquito species Time of dy A B C Mid morning 0.0088 5.4900 5.5000 Mid Afternoon.3400 0.0300 0.8700 Dusk 0.600 5.400 3.000 The Chi squre test sttistic is the sum of
More informationUniversitaireWiskundeCompetitie. Problem 2005/4-A We have k=1. Show that for every q Q satisfying 0 < q < 1, there exists a finite subset K N so that
Problemen/UWC NAW 5/7 nr juni 006 47 Problemen/UWC UniversitireWiskundeCompetitie Edition 005/4 For Session 005/4 we received submissions from Peter Vndendriessche, Vldislv Frnk, Arne Smeets, Jn vn de
More informationAn instructive toy model: two paradoxes
Tel Aviv University, 2006 Gussin rndom vectors 27 3 Level crossings... the fmous ice formul, undoubtedly one of the most importnt results in the ppliction of smooth stochstic processes..j. Adler nd J.E.
More informationMath 426: Probability Final Exam Practice
Mth 46: Probbility Finl Exm Prctice. Computtionl problems 4. Let T k (n) denote the number of prtitions of the set {,..., n} into k nonempty subsets, where k n. Argue tht T k (n) kt k (n ) + T k (n ) by
More informationS. S. Dragomir. 2, we have the inequality. b a
Bull Koren Mth Soc 005 No pp 3 30 SOME COMPANIONS OF OSTROWSKI S INEQUALITY FOR ABSOLUTELY CONTINUOUS FUNCTIONS AND APPLICATIONS S S Drgomir Abstrct Compnions of Ostrowski s integrl ineulity for bsolutely
More informationp(t) dt + i 1 re it ireit dt =
Note: This mteril is contined in Kreyszig, Chpter 13. Complex integrtion We will define integrls of complex functions long curves in C. (This is bit similr to [relvlued] line integrls P dx + Q dy in R2.)
More informationSOLUTIONS FOR ANALYSIS QUALIFYING EXAM, FALL (1 + µ(f n )) f(x) =. But we don t need the exact bound.) Set
SOLUTIONS FOR ANALYSIS QUALIFYING EXAM, FALL 28 Nottion: N {, 2, 3,...}. (Tht is, N.. Let (X, M be mesurble spce with σ-finite positive mesure µ. Prove tht there is finite positive mesure ν on (X, M such
More informationand that at t = 0 the object is at position 5. Find the position of the object at t = 2.
7.2 The Fundmentl Theorem of Clculus 49 re mny, mny problems tht pper much different on the surfce but tht turn out to be the sme s these problems, in the sense tht when we try to pproimte solutions we
More informationf(x)dx . Show that there 1, 0 < x 1 does not exist a differentiable function g : [ 1, 1] R such that g (x) = f(x) for all
3 Definite Integrl 3.1 Introduction In school one comes cross the definition of the integrl of rel vlued function defined on closed nd bounded intervl [, b] between the limits nd b, i.e., f(x)dx s the
More informationThe area under the graph of f and above the x-axis between a and b is denoted by. f(x) dx. π O
1 Section 5. The Definite Integrl Suppose tht function f is continuous nd positive over n intervl [, ]. y = f(x) x The re under the grph of f nd ove the x-xis etween nd is denoted y f(x) dx nd clled the
More informationA HELLY THEOREM FOR FUNCTIONS WITH VALUES IN METRIC SPACES. 1. Introduction
Ttr Mt. Mth. Publ. 44 (29), 159 168 DOI: 1.2478/v1127-9-56-z t m Mthemticl Publictions A HELLY THEOREM FOR FUNCTIONS WITH VALUES IN METRIC SPACES Miloslv Duchoň Peter Mličký ABSTRACT. We present Helly
More informationInfinite Geometric Series
Infinite Geometric Series Finite Geometric Series ( finite SUM) Let 0 < r < 1, nd let n be positive integer. Consider the finite sum It turns out there is simple lgebric expression tht is equivlent to
More informationNumerical Integration
Chpter 5 Numericl Integrtion Numericl integrtion is the study of how the numericl vlue of n integrl cn be found. Methods of function pproximtion discussed in Chpter??, i.e., function pproximtion vi the
More informationON A CONVEXITY PROPERTY. 1. Introduction Most general class of convex functions is defined by the inequality
Krgujevc Journl of Mthemtics Volume 40( (016, Pges 166 171. ON A CONVEXITY PROPERTY SLAVKO SIMIĆ Abstrct. In this rticle we proved n interesting property of the clss of continuous convex functions. This
More informationMAA 4212 Improper Integrals
Notes by Dvid Groisser, Copyright c 1995; revised 2002, 2009, 2014 MAA 4212 Improper Integrls The Riemnn integrl, while perfectly well-defined, is too restrictive for mny purposes; there re functions which
More informationNotes on length and conformal metrics
Notes on length nd conforml metrics We recll how to mesure the Eucliden distnce of n rc in the plne. Let α : [, b] R 2 be smooth (C ) rc. Tht is α(t) (x(t), y(t)) where x(t) nd y(t) re smooth rel vlued
More informationLecture 3: Equivalence Relations
Mthcmp Crsh Course Instructor: Pdric Brtlett Lecture 3: Equivlence Reltions Week 1 Mthcmp 2014 In our lst three tlks of this clss, we shift the focus of our tlks from proof techniques to proof concepts
More informationLecture 14: Quadrature
Lecture 14: Qudrture This lecture is concerned with the evlution of integrls fx)dx 1) over finite intervl [, b] The integrnd fx) is ssumed to be rel-vlues nd smooth The pproximtion of n integrl by numericl
More informationThe Bochner Integral and the Weak Property (N)
Int. Journl of Mth. Anlysis, Vol. 8, 2014, no. 19, 901-906 HIKARI Ltd, www.m-hikri.com http://dx.doi.org/10.12988/ijm.2014.4367 The Bochner Integrl nd the Wek Property (N) Besnik Bush Memetj University
More informationState space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies
Stte spce systems nlysis (continued) Stbility A. Definitions A system is sid to be Asymptoticlly Stble (AS) when it stisfies ut () = 0, t > 0 lim xt () 0. t A system is AS if nd only if the impulse response
More informationLecture 3. Limits of Functions and Continuity
Lecture 3 Limits of Functions nd Continuity Audrey Terrs April 26, 21 1 Limits of Functions Notes I m skipping the lst section of Chpter 6 of Lng; the section bout open nd closed sets We cn probbly live
More informationImproper Integrals. Type I Improper Integrals How do we evaluate an integral such as
Improper Integrls Two different types of integrls cn qulify s improper. The first type of improper integrl (which we will refer to s Type I) involves evluting n integrl over n infinite region. In the grph
More informationthan 1. It means in particular that the function is decreasing and approaching the x-
6 Preclculus Review Grph the functions ) (/) ) log y = b y = Solution () The function y = is n eponentil function with bse smller thn It mens in prticulr tht the function is decresing nd pproching the
More informationThe Wave Equation I. MA 436 Kurt Bryan
1 Introduction The Wve Eqution I MA 436 Kurt Bryn Consider string stretching long the x xis, of indeterminte (or even infinite!) length. We wnt to derive n eqution which models the motion of the string
More informationPhysics 116C Solution of inhomogeneous ordinary differential equations using Green s functions
Physics 6C Solution of inhomogeneous ordinry differentil equtions using Green s functions Peter Young November 5, 29 Homogeneous Equtions We hve studied, especilly in long HW problem, second order liner
More informationMain topics for the First Midterm
Min topics for the First Midterm The Midterm will cover Section 1.8, Chpters 2-3, Sections 4.1-4.8, nd Sections 5.1-5.3 (essentilly ll of the mteril covered in clss). Be sure to know the results of the
More informationThe mth Ratio Convergence Test and Other Unconventional Convergence Tests
The mth Rtio Convergence Test nd Other Unconventionl Convergence Tests Kyle Blckburn My 14, 2012 Contents 1 Introduction 2 2 Definitions, Lemms, nd Theorems 2 2.1 Defintions.............................
More information7.2 The Definite Integral
7.2 The Definite Integrl the definite integrl In the previous section, it ws found tht if function f is continuous nd nonnegtive, then the re under the grph of f on [, b] is given by F (b) F (), where
More informationNew Expansion and Infinite Series
Interntionl Mthemticl Forum, Vol. 9, 204, no. 22, 06-073 HIKARI Ltd, www.m-hikri.com http://dx.doi.org/0.2988/imf.204.4502 New Expnsion nd Infinite Series Diyun Zhng College of Computer Nnjing University
More information(e) if x = y + z and a divides any two of the integers x, y, or z, then a divides the remaining integer
Divisibility In this note we introduce the notion of divisibility for two integers nd b then we discuss the division lgorithm. First we give forml definition nd note some properties of the division opertion.
More informationProperties of the Riemann Integral
Properties of the Riemnn Integrl Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University Februry 15, 2018 Outline 1 Some Infimum nd Supremum Properties 2
More informationGoals: Determine how to calculate the area described by a function. Define the definite integral. Explore the relationship between the definite
Unit #8 : The Integrl Gols: Determine how to clculte the re described by function. Define the definite integrl. Eplore the reltionship between the definite integrl nd re. Eplore wys to estimte the definite
More informationCoalgebra, Lecture 15: Equations for Deterministic Automata
Colger, Lecture 15: Equtions for Deterministic Automt Julin Slmnc (nd Jurrin Rot) Decemer 19, 2016 In this lecture, we will study the concept of equtions for deterministic utomt. The notes re self contined
More informationCS667 Lecture 6: Monte Carlo Integration 02/10/05
CS667 Lecture 6: Monte Crlo Integrtion 02/10/05 Venkt Krishnrj Lecturer: Steve Mrschner 1 Ide The min ide of Monte Crlo Integrtion is tht we cn estimte the vlue of n integrl by looking t lrge number of
More informationDIRECT CURRENT CIRCUITS
DRECT CURRENT CUTS ELECTRC POWER Consider the circuit shown in the Figure where bttery is connected to resistor R. A positive chrge dq will gin potentil energy s it moves from point to point b through
More informationA BRIEF INTRODUCTION TO UNIFORM CONVERGENCE. In the study of Fourier series, several questions arise naturally, such as: c n e int
A BRIEF INTRODUCTION TO UNIFORM CONVERGENCE HANS RINGSTRÖM. Questions nd exmples In the study of Fourier series, severl questions rise nturlly, such s: () (2) re there conditions on c n, n Z, which ensure
More informationBest Approximation in the 2-norm
Jim Lmbers MAT 77 Fll Semester 1-11 Lecture 1 Notes These notes correspond to Sections 9. nd 9.3 in the text. Best Approximtion in the -norm Suppose tht we wish to obtin function f n (x) tht is liner combintion
More informationThe use of a so called graphing calculator or programmable calculator is not permitted. Simple scientific calculators are allowed.
ERASMUS UNIVERSITY ROTTERDAM Informtion concerning the Entrnce exmintion Mthemtics level 1 for Interntionl Bchelor in Communiction nd Medi Generl informtion Avilble time: 2 hours 30 minutes. The exmintion
More informationThe Riemann-Lebesgue Lemma
Physics 215 Winter 218 The Riemnn-Lebesgue Lemm The Riemnn Lebesgue Lemm is one of the most importnt results of Fourier nlysis nd symptotic nlysis. It hs mny physics pplictions, especilly in studies of
More informationFarey Fractions. Rickard Fernström. U.U.D.M. Project Report 2017:24. Department of Mathematics Uppsala University
U.U.D.M. Project Report 07:4 Frey Frctions Rickrd Fernström Exmensrete i mtemtik, 5 hp Hledre: Andres Strömergsson Exmintor: Jörgen Östensson Juni 07 Deprtment of Mthemtics Uppsl University Frey Frctions
More informationChapter 22. The Fundamental Theorem of Calculus
Version of 24.2.4 Chpter 22 The Fundmentl Theorem of Clculus In this chpter I ddress one of the most importnt properties of the Lebesgue integrl. Given n integrble function f : [,b] R, we cn form its indefinite
More informationIntroduction to Group Theory
Introduction to Group Theory Let G be n rbitrry set of elements, typiclly denoted s, b, c,, tht is, let G = {, b, c, }. A binry opertion in G is rule tht ssocites with ech ordered pir (,b) of elements
More informationA NOTE ON PREPARACOMPACTNESS
Volume 1, 1976 Pges 253 260 http://topology.uburn.edu/tp/ A NOTE ON PREPARACOMPACTNE by J. C. mith Topology Proceedings Web: http://topology.uburn.edu/tp/ Mil: Topology Proceedings Deprtment of Mthemtics
More informationThe final exam will take place on Friday May 11th from 8am 11am in Evans room 60.
Mth 104: finl informtion The finl exm will tke plce on Fridy My 11th from 8m 11m in Evns room 60. The exm will cover ll prts of the course with equl weighting. It will cover Chpters 1 5, 7 15, 17 21, 23
More informationLecture 3 ( ) (translated and slightly adapted from lecture notes by Martin Klazar)
Lecture 3 (5.3.2018) (trnslted nd slightly dpted from lecture notes by Mrtin Klzr) Riemnn integrl Now we define precisely the concept of the re, in prticulr, the re of figure U(, b, f) under the grph of
More informationMath 6455 Oct 10, Differential Geometry I Fall 2006, Georgia Tech
Mth 6455 Oct 10, 2006 1 Differentil Geometry I Fll 2006, Georgi Tech Lecture Notes 12 Riemnnin Metrics 0.1 Definition If M is smooth mnifold then by Riemnnin metric g on M we men smooth ssignment of n
More information4 7x =250; 5 3x =500; Read section 3.3, 3.4 Announcements: Bell Ringer: Use your calculator to solve
Dte: 3/14/13 Objective: SWBAT pply properties of exponentil functions nd will pply properties of rithms. Bell Ringer: Use your clcultor to solve 4 7x =250; 5 3x =500; HW Requests: Properties of Log Equtions
More information