MEASURE-THEORETIC ENTROPY
|
|
- Toby Hodges
- 5 years ago
- Views:
Transcription
1 MEASURE-THEORETIC ENTROPY Abstract. We introduce measure-theoretic entropy 1. Some motivation for the formula and the logs We want to define a function I : [0, 1] R which measures how suprised we are or how much information we get when we observe random events, so that if an event occurs with probability p, we want I(p) to be the measure of our surprise. For example, if X Bern(0.01), we would be surprised if we observe X = 1, and not so much if we observe X = 0. By making the following assumptions on I, we can determine I up to a constant. (a) I : [0, 1] [0, ). (b) For all p, q [0, 1], we have I(pq) = I(p) + I(q). (c) I is continuous. To motivate condition (b), suppose that the events A and B are independent and occur with probabilities p and q, respectively. It seems reasonable that information should be additive for independent events, so we require (b) holds. Proposition 1. The assumptions on I give that for some constant k > 0, we have that I(p) = k log p, where log = ln. Exercise 2. Prove Proposition Entropy with respect to a partition Let (Ω, F, µ) be a probability space. Let P be a finite (measurable partition) of Ω; that is, α F, the union of the elements of α is all of Ω, and the measure of the intersection of any two distinct elements of α is zero. Given ω Ω, we let α(ω) be the part that ω belongs to. The information function with respect to α is given by I α (ω) := log µ(α(ω)) = A α log(µ(a))1 A (ω). The static entropy of (Ω, F, µ) with respect to the partition α is defined to be the non-negative number given by H µ (α) = H(α) := EI α = A α µ(a) log(µ(a)). 1
2 2 MEASURE-THEORETIC ENTROPY Here we take the convention that 0(log 0) = 0. Given two partitions α = {A 1,..., A n } and β = {B 1,..., B m } of Ω, we say that β refines α if every element of β is a subset of an element of α. We let α β denote the least common refinement of α and β given by α β := {A i B j : 1 i n, 1 j m}. Example 3 (i.i.d systems). Let p be a probability measure on a finite alphabet A = {a 1,..., a n }. Consider the probability space (A Z, F, µ), where F is the usual product sigma-algebra and µ = p Z. Let P i = { x A Z : x 0 = a i } and P 0 = {P 1,..., P n }. We have that H µ (P 0 ) = H(p). Exercise 4. Let (Ω, F, µ) be a probability space. Let α and β be partitions of Ω. Show that if β refines α, then H µ (β) H µ (α). Given a probability vector p = (p 1,..., p n ), we also write H(p) = p 1 log p i, and if X is a random variable with pdf p, we also write H(X) = H(p). Example 5 (i.i.d. systems continued). Let T be the left-shift. Define P 1 = T 1 P 0 := {T 1 P 1,..., T 1 P n }. Note that T 1 P i = { x A Z : x 1 = a i } and P 1 is also a partition of A Z. We have that H µ (P 0 P 1 ) = p j p i log(p i p j ) = p i p j log p i + p i p j log p j = 2H(p) We say that two partitions α and β are independent if µ(a B) = µ(a)µ(b) for all A α and B α. Exercise 6. Let (Ω, F, µ) be a probability space. Let α and β be independent partitions of Ω. Show that H(α β) = H(α) + H(β). Exercise 7 (Markov chains). Let X be a Markov chain which takes values on the state space {0, 1} with a transition matrix P, given by p 00 = 1, p 2 01 = 1, p 2 10 = 1, and p 4 11 = 3. Find π the stationary 4 distribution for P. Let Ω = {0, 1} N, and P 0 and P 1 be defined as before. Let µ( ) = P(X ). Compute H(P 0 P 1 ).
3 MEASURE-THEORETIC ENTROPY 3 3. Dynamical entropy with respect to a partition Lemma 8. Let (Ω, F, µ) be a probability space. If α and β be partitions of Ω, then H(α β) H(α) + H(β). We will prove Lemma 8 using the following useful inequality. Proposition 9 (Gibbs inequality). Let p = (p 1,..., p n ) be a probability vector. If q = (q 1,..., q n ) is another probability vector then we define H(p) = p i log p i p i log q i, where equality holds iff p = q. Exercise 10. Use Jensen s inequality to prove Proposition 9. Proof of Lemma 8. Let r(i, j) = µ(a j B i ), p(i) = µ(a i ) and q(j) = µ(b j ). Let MI(r) := ( r(i, j) ) r(i, j) log. p(i)q(j) By Proposition 9, we have that MI(r) 0. On the other hand, MI(r) = H(p) + H(q) H(r), thus H(α β) + MI(r) H(α) + H(β). We say that a sequence of real numbers a n is subaddtive if a n+m a n + a m. Exercise 11 (Subadditive sequences). If a n is a subadditive sequence of real numbers, then lim n a n /n = inf a n /n. Exercise 12. Let (X i ) i N be a sequence of i.i.d. random variables. Show that a n = log P(S n > n) is subadditive, where S n = X X n. Let (Ω, F, µ) be a probability space, and T be a measure-preserving map. Let α be a partition. Consider the sequence given by a n := H(α T 1 α T n+1 α). (1) It is not difficult to show that a n is subadditive. Hence we may define the dynamical entropy of (Ω, F, µ, T ) with respect to the partition α to be a n h µ (α, T ) := lim n n.
4 4 MEASURE-THEORETIC ENTROPY The dynamical entropy of (Ω, F, µ, T ) is defined h µ (T ) := sup h µ (α, T ), α where the supremum is taken over all measurable partitions. Most of time this is just referred to as the entropy or the Kolmogorov-Sinai entropy. Exercise 13. Show that the sequence given in (1) is subadditive. Exercise 14. Let (Ω, F, µ, T ) be a measure-preserving system. Suppose that (Ω, F, µ, T ) is a factor. Show that h µ (T ) h µ (T ). Conclude that entropy is an isomorphism invariant. Exercise 15. Show that the Kolmogorov-Sinai entropy of a Bernoulli shift of N symbols is bounded. Exercise 16 (Infinite entropy). Let λ be Lebesgue measure on [0, 1]. Consider the space of all sequences taking values in [0, 1], given by ([0, 1] N, F, λ N, T ), where T is the left-shift, and F is the product sigmaalgebra. Show that this space has infinite Kolmogorov-Sinain entropy. 4. Generating partitions Let (Ω, F, µ, T ) be an invertible measure-preserving system. Let α be a finite partition. We say that α is a generating partition if the sigma-algebra generated by α, T 1 α, T 1 α, T 2 α, T 2 α,... is all of F; we say that α is a one-sided generating partition if α, T 1 α,... generates all of F. Theorem 17 (Kolmogorov-Sinai). Let (Ω, F, µ, T ) be a measure-preserving system. If α is a generating partition, then h µ (α, T ) = h µ (T ). Exercise 18 (iid systems again). Notice that in Example 5, P 0 is a generating partition. By Exercise 6, we have that H µ (P 0 T n 1 P 0 ) = nh(p 0 ), so that h µ (T ) = H(p). Exercise 19. Compute the entropy for Markov chain in Example 7 (endowed with the left-shift). Theorem 20 (Rokhlin s countable generator theorem). Let (Ω, F, µ, T ) be an invertible ergodic measure-preserving system. If h µ (T ) <, then there exists a countable generating partition α for the system, where H µ (α) <. Theorem 21 (Krieger finite generator theorem). Let (Ω, F, µ, T ) be an invertible ergodic measure-preserving system. If h µ (T ) <, then there exists a finite generating partition for the system.
5 MEASURE-THEORETIC ENTROPY 5 Theorems 20 and 21 make good presentation topics. The original proofs of these theorems can be quite difficult, however there are easier proofs due to Keane and Serafin. Exercise 22. Let (Ω, F, µ, T ) be an invertible ergodic measure-preserving system. If h µ (T ) <, show that there exist N > 0 and a bijective map φ : Ω [N] Z such that φ T = S φ, where S is the left-shift. Remark 23 (A word of caution for non-invertible systems). Theorem 21 may not hold if T is not invertible. Although, Sinai proved that every ergodic measure-preserving system, whether invertible or not, has all Bernoulli shifts of no greater entropy has factors, the isomorphism theorem of Ornstein may not hold for one-sided Bernoulli shifts. Although, entropy is still an invariant, it is not a complete invariant for one-sided Bernoulli shifts. One can see this by considering Melshalkin s example and counting pre-images. Presentation topic 24. Present an overview of what can and can not be done in the one-sided case. References [1] T. Downarowicz. Entropy in dynamical systems, volume 18 of New Mathematical Monographs. Cambridge University Press, Cambridge, [2] R. W. Hamming. Coding and information theory. Prentice-Hall, Inc., Englewood Cliffs, N.J., [3] C. Hoffman and D. Rudolph. Uniform endomorphisms which are isomorphic to a bernoulli shift. Annals of Mathematics, 156(1):pp , [4] M. S. Keane and J. Serafin. On the countable generator theorem. Fund. Math., 157(2-3): , Dedicated to the memory of Wies law Szlenk. [5] M. S. Keane and J. Serafin. Generators. Period. Math. Hungar., 44(2): , [6] W. Parry and P. Walters. Endomorphisms of a lebesgue space. Bull. Amer. Math. Soc., 78(2): , [7] K. Petersen. Ergodic theory, volume 2 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, Corrected reprint of the 1983 original.
MAGIC010 Ergodic Theory Lecture Entropy
7. Entropy 7. Introduction A natural question in mathematics is the so-called isomorphism problem : when are two mathematical objects of the same class the same (in some appropriately defined sense of
More informationA dyadic endomorphism which is Bernoulli but not standard
A dyadic endomorphism which is Bernoulli but not standard Christopher Hoffman Daniel Rudolph November 4, 2005 Abstract Any measure preserving endomorphism generates both a decreasing sequence of σ-algebras
More informationJOININGS, FACTORS, AND BAIRE CATEGORY
JOININGS, FACTORS, AND BAIRE CATEGORY Abstract. We discuss the Burton-Rothstein approach to Ornstein theory. 1. Weak convergence Let (X, B) be a metric space and B be the Borel sigma-algebra generated
More informationAn Introduction to Entropy and Subshifts of. Finite Type
An Introduction to Entropy and Subshifts of Finite Type Abby Pekoske Department of Mathematics Oregon State University pekoskea@math.oregonstate.edu August 4, 2015 Abstract This work gives an overview
More informationChapter 4. Measure Theory. 1. Measure Spaces
Chapter 4. Measure Theory 1. Measure Spaces Let X be a nonempty set. A collection S of subsets of X is said to be an algebra on X if S has the following properties: 1. X S; 2. if A S, then A c S; 3. if
More informationErgodic Theorems. Samy Tindel. Purdue University. Probability Theory 2 - MA 539. Taken from Probability: Theory and examples by R.
Ergodic Theorems Samy Tindel Purdue University Probability Theory 2 - MA 539 Taken from Probability: Theory and examples by R. Durrett Samy T. Ergodic theorems Probability Theory 1 / 92 Outline 1 Definitions
More information4. Ergodicity and mixing
4. Ergodicity and mixing 4. Introduction In the previous lecture we defined what is meant by an invariant measure. In this lecture, we define what is meant by an ergodic measure. The primary motivation
More informationInformation Theory and Statistics Lecture 3: Stationary ergodic processes
Information Theory and Statistics Lecture 3: Stationary ergodic processes Łukasz Dębowski ldebowsk@ipipan.waw.pl Ph. D. Programme 2013/2014 Measurable space Definition (measurable space) Measurable space
More informationKrieger s finite generator theorem for ergodic actions of countable groups
Krieger s finite generator theorem for ergodic actions of countable groups Brandon Seward University of Michigan UNT RTG Conference Logic, Dynamics, and Their Interactions II June 6, 2014 Brandon Seward
More informationNaive entropy of dynamical systems
Naive entropy of dynamical systems Peter Burton March, 06 Abstract We study an invariant of dynamical systems called naive entropy, which is defined for both measurable and topological actions of any countable
More informationwhich element of ξ contains the outcome conveys all the available information.
CHAPTER III MEASURE THEORETIC ENTROPY In this chapter we define the measure theoretic entropy or metric entropy of a measure preserving transformation. Equalities are to be understood modulo null sets;
More informationUniform mixing and completely positive sofic entropy
Uniform mixing and completely positive sofic entropy Tim Austin and Peter Burton October 3, 206 Abstract Let G be a countable discrete sofic group. We define a concept of uniform mixing for measurepreserving
More informationIf the [T, Id] automorphism is Bernoulli then the [T, Id] endomorphism is standard
If the [T, Id] automorphism is Bernoulli then the [T, Id] endomorphism is standard Christopher Hoffman Daniel Rudolph September 27, 2002 Abstract For any - measure preserving map T of a probability space
More informationBERNOULLI ACTIONS OF SOFIC GROUPS HAVE COMPLETELY POSITIVE ENTROPY
BERNOULLI ACTIONS OF SOFIC GROUPS HAVE COMPLETELY POSITIVE ENTROPY DAVID KERR Abstract. We prove that every Bernoulli action of a sofic group has completely positive entropy with respect to every sofic
More informationTHE MEASURE-THEORETIC ENTROPY OF LINEAR CELLULAR AUTOMATA WITH RESPECT TO A MARKOV MEASURE. 1. Introduction
THE MEASURE-THEORETIC ENTROPY OF LINEAR CELLULAR AUTOMATA WITH RESPECT TO A MARKOV MEASURE HASAN AKIN Abstract. The purpose of this short paper is to compute the measure-theoretic entropy of the onedimensional
More informationTAIL FIELDS GENERATED BY SYMBOL COUNTS IN MEASURE-PRESERVING SYSTEMS
TAIL FIELDS GENERATED BY SYMBOL COUNTS IN MEASURE-PRESERVING SYSTEMS KARL PETERSEN AND JEAN-PAUL THOUVENOT In memory of Anzelm Iwanik Abstract. A finite-state stationary process is called (one or twosided)
More informationLecture 3: Probability Measures - 2
Lecture 3: Probability Measures - 2 1. Continuation of measures 1.1 Problem of continuation of a probability measure 1.2 Outer measure 1.3 Lebesgue outer measure 1.4 Lebesgue continuation of an elementary
More informationVARIATIONAL PRINCIPLE FOR THE ENTROPY
VARIATIONAL PRINCIPLE FOR THE ENTROPY LUCIAN RADU. Metric entropy Let (X, B, µ a measure space and I a countable family of indices. Definition. We say that ξ = {C i : i I} B is a measurable partition if:
More informationNotes on Measure, Probability and Stochastic Processes. João Lopes Dias
Notes on Measure, Probability and Stochastic Processes João Lopes Dias Departamento de Matemática, ISEG, Universidade de Lisboa, Rua do Quelhas 6, 1200-781 Lisboa, Portugal E-mail address: jldias@iseg.ulisboa.pt
More informationRudiments of Ergodic Theory
Rudiments of Ergodic Theory Zefeng Chen September 24, 203 Abstract In this note we intend to present basic ergodic theory. We begin with the notion of a measure preserving transformation. We then define
More informationII - REAL ANALYSIS. This property gives us a way to extend the notion of content to finite unions of rectangles: we define
1 Measures 1.1 Jordan content in R N II - REAL ANALYSIS Let I be an interval in R. Then its 1-content is defined as c 1 (I) := b a if I is bounded with endpoints a, b. If I is unbounded, we define c 1
More informationINTRODUCTION TO FURSTENBERG S 2 3 CONJECTURE
INTRODUCTION TO FURSTENBERG S 2 3 CONJECTURE BEN CALL Abstract. In this paper, we introduce the rudiments of ergodic theory and entropy necessary to study Rudolph s partial solution to the 2 3 problem
More information18.175: Lecture 2 Extension theorems, random variables, distributions
18.175: Lecture 2 Extension theorems, random variables, distributions Scott Sheffield MIT Outline Extension theorems Characterizing measures on R d Random variables Outline Extension theorems Characterizing
More information1.1. MEASURES AND INTEGRALS
CHAPTER 1: MEASURE THEORY In this chapter we define the notion of measure µ on a space, construct integrals on this space, and establish their basic properties under limits. The measure µ(e) will be defined
More informationLecture 4. Entropy and Markov Chains
preliminary version : Not for diffusion Lecture 4. Entropy and Markov Chains The most important numerical invariant related to the orbit growth in topological dynamical systems is topological entropy.
More information1.4 Outer measures 10 CHAPTER 1. MEASURE
10 CHAPTER 1. MEASURE 1.3.6. ( Almost everywhere and null sets If (X, A, µ is a measure space, then a set in A is called a null set (or µ-null if its measure is 0. Clearly a countable union of null sets
More informationOverview of Markovian maps
Overview of Markovian maps Mike Boyle University of Maryland BIRS October 2007 *these slides are online, via Google: Mike Boyle Outline of the talk I. Subshift background 1. Subshifts 2. (Sliding) block
More informationThe Lebesgue Integral
The Lebesgue Integral Brent Nelson In these notes we give an introduction to the Lebesgue integral, assuming only a knowledge of metric spaces and the iemann integral. For more details see [1, Chapters
More information3 hours UNIVERSITY OF MANCHESTER. 22nd May and. Electronic calculators may be used, provided that they cannot store text.
3 hours MATH40512 UNIVERSITY OF MANCHESTER DYNAMICAL SYSTEMS AND ERGODIC THEORY 22nd May 2007 9.45 12.45 Answer ALL four questions in SECTION A (40 marks in total) and THREE of the four questions in SECTION
More informationKRIEGER S FINITE GENERATOR THEOREM FOR ACTIONS OF COUNTABLE GROUPS I
KRIEGER S FINITE GENERATOR THEOREM FOR ACTIONS OF COUNTABLE GROUPS I BRANDON SEWARD Abstract. For an ergodic p.m.p. action G (X, µ of a countable group G, we define the Rokhlin entropy h Rok G (X, µ to
More informationABSTRACT CONDITIONAL EXPECTATION IN L 2
ABSTRACT CONDITIONAL EXPECTATION IN L 2 Abstract. We prove that conditional expecations exist in the L 2 case. The L 2 treatment also gives us a geometric interpretation for conditional expectation. 1.
More informationChapter 8. General Countably Additive Set Functions. 8.1 Hahn Decomposition Theorem
Chapter 8 General Countably dditive Set Functions In Theorem 5.2.2 the reader saw that if f : X R is integrable on the measure space (X,, µ) then we can define a countably additive set function ν on by
More informationIRRATIONAL ROTATION OF THE CIRCLE AND THE BINARY ODOMETER ARE FINITARILY ORBIT EQUIVALENT
IRRATIONAL ROTATION OF THE CIRCLE AND THE BINARY ODOMETER ARE FINITARILY ORBIT EQUIVALENT MRINAL KANTI ROYCHOWDHURY Abstract. Two invertible dynamical systems (X, A, µ, T ) and (Y, B, ν, S) where X, Y
More informationProbability and Measure
Part II Year 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2018 84 Paper 4, Section II 26J Let (X, A) be a measurable space. Let T : X X be a measurable map, and µ a probability
More informationLECTURE 3. Last time:
LECTURE 3 Last time: Mutual Information. Convexity and concavity Jensen s inequality Information Inequality Data processing theorem Fano s Inequality Lecture outline Stochastic processes, Entropy rate
More informationThe Hopf argument. Yves Coudene. IRMAR, Université Rennes 1, campus beaulieu, bat Rennes cedex, France
The Hopf argument Yves Coudene IRMAR, Université Rennes, campus beaulieu, bat.23 35042 Rennes cedex, France yves.coudene@univ-rennes.fr slightly updated from the published version in Journal of Modern
More informationABSTRACT INTEGRATION CHAPTER ONE
CHAPTER ONE ABSTRACT INTEGRATION Version 1.1 No rights reserved. Any part of this work can be reproduced or transmitted in any form or by any means. Suggestions and errors are invited and can be mailed
More informationAbstract This paper introduces a new measure-conjugacy invariant for actions of free groups. Using this invariant, it is shown that two Bernoulli
Abstract This paper introduces a new measure-conjugacy invariant for actions of free groups. Using this invariant, it is shown that two Bernoulli shifts over a finitely generated free group are measurably
More informationOn a measurable analogue of small topological full groups
On a measurable analogue of small topological full groups François Le Maître November 5, 2018 arxiv:1608.07399v2 [math.ds] 7 May 2018 Abstract We initiate the study of a measurable analogue of small topological
More informationA ZERO ENTROPY T SUCH THAT THE [T,ID] ENDOMORPHISM IS NONSTANDARD
A ZERO ENTROPY T SUCH THAT THE [T,ID] ENDOMORPHISM IS NONSTANDARD CHRISTOPHER HOFFMAN Abstract. We present an example of an ergodic transformation T, a variant of a zero entropy non loosely Bernoulli map
More informationA topological semigroup structure on the space of actions modulo weak equivalence.
A topological semigroup structure on the space of actions modulo wea equivalence. Peter Burton January 8, 08 Abstract We introduce a topology on the space of actions modulo wea equivalence finer than the
More informationRotation set for maps of degree 1 on sun graphs. Sylvie Ruette. January 6, 2019
Rotation set for maps of degree 1 on sun graphs Sylvie Ruette January 6, 2019 Abstract For a continuous map on a topological graph containing a unique loop S, it is possible to define the degree and, for
More informationarxiv: v2 [math.ds] 26 Oct 2008
Abstract This paper introduces a new measure-conjugacy invariant for actions of free groups. Using this invariant, it is shown that two Bernoulli shifts over a finitely generated free group are measurably
More informationarxiv: v1 [math.ds] 4 Jan 2019
EPLODING MARKOV OPERATORS BARTOSZ FREJ ariv:1901.01329v1 [math.ds] 4 Jan 2019 Abstract. A special class of doubly stochastic (Markov operators is constructed. These operators come from measure preserving
More informationarxiv: v1 [math.ds] 20 Feb 2015
On Pinsker factors for Rokhlin entropy Andrei Alpeev arxiv:1502.06036v1 [math.ds] 20 Feb 2015 February 24, 2015 Abstract In this paper we will prove that any dynamical system posess the unique maximal
More informationKRIEGER S FINITE GENERATOR THEOREM FOR ACTIONS OF COUNTABLE GROUPS II
KRIEER S FINITE ENERATOR THEOREM FOR ACTIONS OF COUNTABLE ROUPS II BRANDON SEWARD Abstract. We continue the study of Rokhlin entropy, an isomorphism invariant for p.m.p. actions of countable groups introduced
More informationA loosely Bernoulli counterexample machine
A loosely Bernoulli counterexample machine Christopher Hoffman September 7, 00 Abstract In Rudolph s paper on minimal self joinings [7] he proves that a rank one mixing transformation constructed by Ornstein
More informationarxiv: v1 [math.co] 5 Apr 2019
arxiv:1904.02924v1 [math.co] 5 Apr 2019 The asymptotics of the partition of the cube into Weyl simplices, and an encoding of a Bernoulli scheme A. M. Vershik 02.02.2019 Abstract We suggest a combinatorial
More informationLebesgue measure on R is just one of many important measures in mathematics. In these notes we introduce the general framework for measures.
Measures In General Lebesgue measure on R is just one of many important measures in mathematics. In these notes we introduce the general framework for measures. Definition: σ-algebra Let X be a set. A
More informationThe theory of Bernoulli Shifts
The theory of Bernoulli Shifts by Paul C. Shields Professor Emeritus of Mathematics University of Toledo, Toledo, OH 4366 Send corrections to bshifts@earthlink.net Web Edition 1.1 Reprint of 1973 University
More informationErgodic Theory. Constantine Caramanis. May 6, 1999
Ergodic Theory Constantine Caramanis ay 6, 1999 1 Introduction Ergodic theory involves the study of transformations on measure spaces. Interchanging the words measurable function and probability density
More informationLebesgue Integration: A non-rigorous introduction. What is wrong with Riemann integration?
Lebesgue Integration: A non-rigorous introduction What is wrong with Riemann integration? xample. Let f(x) = { 0 for x Q 1 for x / Q. The upper integral is 1, while the lower integral is 0. Yet, the function
More informationChapter IV Integration Theory
Chapter IV Integration Theory This chapter is devoted to the developement of integration theory. The main motivation is to extend the Riemann integral calculus to larger types of functions, thus leading
More informationLebesgue Measure. Dung Le 1
Lebesgue Measure Dung Le 1 1 Introduction How do we measure the size of a set in IR? Let s start with the simplest ones: intervals. Obviously, the natural candidate for a measure of an interval is its
More informationPart II Probability and Measure
Part II Probability and Measure Theorems Based on lectures by J. Miller Notes taken by Dexter Chua Michaelmas 2016 These notes are not endorsed by the lecturers, and I have modified them (often significantly)
More information1 Stat 605. Homework I. Due Feb. 1, 2011
The first part is homework which you need to turn in. The second part is exercises that will not be graded, but you need to turn it in together with the take-home final exam. 1 Stat 605. Homework I. Due
More informationIntegration on Measure Spaces
Chapter 3 Integration on Measure Spaces In this chapter we introduce the general notion of a measure on a space X, define the class of measurable functions, and define the integral, first on a class of
More informationInformation Theory. Lecture 5 Entropy rate and Markov sources STEFAN HÖST
Information Theory Lecture 5 Entropy rate and Markov sources STEFAN HÖST Universal Source Coding Huffman coding is optimal, what is the problem? In the previous coding schemes (Huffman and Shannon-Fano)it
More informationSpring 2014 Advanced Probability Overview. Lecture Notes Set 1: Course Overview, σ-fields, and Measures
36-752 Spring 2014 Advanced Probability Overview Lecture Notes Set 1: Course Overview, σ-fields, and Measures Instructor: Jing Lei Associated reading: Sec 1.1-1.4 of Ash and Doléans-Dade; Sec 1.1 and A.1
More informationMeasures. 1 Introduction. These preliminary lecture notes are partly based on textbooks by Athreya and Lahiri, Capinski and Kopp, and Folland.
Measures These preliminary lecture notes are partly based on textbooks by Athreya and Lahiri, Capinski and Kopp, and Folland. 1 Introduction Our motivation for studying measure theory is to lay a foundation
More informationEntropy dimensions and a class of constructive examples
Entropy dimensions and a class of constructive examples Sébastien Ferenczi Institut de Mathématiques de Luminy CNRS - UMR 6206 Case 907, 63 av. de Luminy F3288 Marseille Cedex 9 (France) and Fédération
More information1 Measurable Functions
36-752 Advanced Probability Overview Spring 2018 2. Measurable Functions, Random Variables, and Integration Instructor: Alessandro Rinaldo Associated reading: Sec 1.5 of Ash and Doléans-Dade; Sec 1.3 and
More informationµ (X) := inf l(i k ) where X k=1 I k, I k an open interval Notice that is a map from subsets of R to non-negative number together with infinity
A crash course in Lebesgue measure theory, Math 317, Intro to Analysis II These lecture notes are inspired by the third edition of Royden s Real analysis. The Jordan content is an attempt to extend the
More informationSerena Doria. Department of Sciences, University G.d Annunzio, Via dei Vestini, 31, Chieti, Italy. Received 7 July 2008; Revised 25 December 2008
Journal of Uncertain Systems Vol.4, No.1, pp.73-80, 2010 Online at: www.jus.org.uk Different Types of Convergence for Random Variables with Respect to Separately Coherent Upper Conditional Probabilities
More informationChapter 1 Measure-Theoretic Entropy, Introduction
Chapter 1 Measure-Theoretic Entropy, Introduction... nobody knows what entropy really is, so in a debate you will always have the advantage. attr. von Neumann LetX, B,µbeaprobabilityspace,andletT : X X
More informationMeasure and integration
Chapter 5 Measure and integration In calculus you have learned how to calculate the size of different kinds of sets: the length of a curve, the area of a region or a surface, the volume or mass of a solid.
More informationReview of measure theory
209: Honors nalysis in R n Review of measure theory 1 Outer measure, measure, measurable sets Definition 1 Let X be a set. nonempty family R of subsets of X is a ring if, B R B R and, B R B R hold. bove,
More informationDisintegration into conditional measures: Rokhlin s theorem
Disintegration into conditional measures: Rokhlin s theorem Let Z be a compact metric space, µ be a Borel probability measure on Z, and P be a partition of Z into measurable subsets. Let π : Z P be the
More informationSTAT 7032 Probability Spring Wlodek Bryc
STAT 7032 Probability Spring 2018 Wlodek Bryc Created: Friday, Jan 2, 2014 Revised for Spring 2018 Printed: January 9, 2018 File: Grad-Prob-2018.TEX Department of Mathematical Sciences, University of Cincinnati,
More information2 Probability, random elements, random sets
Tel Aviv University, 2012 Measurability and continuity 25 2 Probability, random elements, random sets 2a Probability space, measure algebra........ 25 2b Standard models................... 30 2c Random
More information25.1 Ergodicity and Metric Transitivity
Chapter 25 Ergodicity This lecture explains what it means for a process to be ergodic or metrically transitive, gives a few characterizes of these properties (especially for AMS processes), and deduces
More informationSYMBOLIC DYNAMICS FOR HYPERBOLIC SYSTEMS. 1. Introduction (30min) We want to find simple models for uniformly hyperbolic systems, such as for:
SYMBOLIC DYNAMICS FOR HYPERBOLIC SYSTEMS YURI LIMA 1. Introduction (30min) We want to find simple models for uniformly hyperbolic systems, such as for: [ ] 2 1 Hyperbolic toral automorphisms, e.g. f A
More informationMATS113 ADVANCED MEASURE THEORY SPRING 2016
MATS113 ADVANCED MEASURE THEORY SPRING 2016 Foreword These are the lecture notes for the course Advanced Measure Theory given at the University of Jyväskylä in the Spring of 2016. The lecture notes can
More informationThe first half century of entropy: the most glorious number in dynamics
The first half century of entropy: the most glorious number in dynamics A. Katok Penn State University This is an expanded version of the invited talk given on June 17, 2003 in Moscow at the conference
More informationThe Caratheodory Construction of Measures
Chapter 5 The Caratheodory Construction of Measures Recall how our construction of Lebesgue measure in Chapter 2 proceeded from an initial notion of the size of a very restricted class of subsets of R,
More informationEntropy and Ergodic Theory Notes 22: The Kolmogorov Sinai entropy of a measure-preserving system
Entropy and Ergodic Theory Notes 22: The Kolmogorov Sinai entropy of a measure-preserving system 1 Joinings and channels 1.1 Joinings Definition 1. If px, µ, T q and py, ν, Sq are MPSs, then a joining
More informationMATH5011 Real Analysis I. Exercise 1 Suggested Solution
MATH5011 Real Analysis I Exercise 1 Suggested Solution Notations in the notes are used. (1) Show that every open set in R can be written as a countable union of mutually disjoint open intervals. Hint:
More informationEntropy and Large Deviations
Entropy and Large Deviations p. 1/32 Entropy and Large Deviations S.R.S. Varadhan Courant Institute, NYU Michigan State Universiy East Lansing March 31, 2015 Entropy comes up in many different contexts.
More informationU e = E (U\E) e E e + U\E e. (1.6)
12 1 Lebesgue Measure 1.2 Lebesgue Measure In Section 1.1 we defined the exterior Lebesgue measure of every subset of R d. Unfortunately, a major disadvantage of exterior measure is that it does not satisfy
More informationRandom Process Lecture 1. Fundamentals of Probability
Random Process Lecture 1. Fundamentals of Probability Husheng Li Min Kao Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville Spring, 2016 1/43 Outline 2/43 1 Syllabus
More informationMTH 404: Measure and Integration
MTH 404: Measure and Integration Semester 2, 2012-2013 Dr. Prahlad Vaidyanathan Contents I. Introduction....................................... 3 1. Motivation................................... 3 2. The
More informationDigit Frequencies and Bernoulli Convolutions
Digit Frequencies and Bernoulli Convolutions Tom Kempton November 22, 202 arxiv:2.5023v [math.ds] 2 Nov 202 Abstract It is well known that the Bernoulli convolution ν associated to the golden mean has
More informationFINITE ENTROPY ACTIONS OF FREE GROUPS, RIGIDITY OF STABILIZERS, AND A HOWE MOORE TYPE PHENOMENON
FINITE ENTROPY ACTIONS OF FREE GROUPS, RIGIDITY OF STABILIZERS, AND A HOWE MOORE TYPE PHENOMENON BRANDON SEWARD Abstract. We study a notion of entropy for probability measure preserving actions of finitely
More informationComplex Systems Methods 2. Conditional mutual information, entropy rate and algorithmic complexity
Complex Systems Methods 2. Conditional mutual information, entropy rate and algorithmic complexity Eckehard Olbrich MPI MiS Leipzig Potsdam WS 2007/08 Olbrich (Leipzig) 26.10.2007 1 / 18 Overview 1 Summary
More informationRS Chapter 1 Random Variables 6/5/2017. Chapter 1. Probability Theory: Introduction
Chapter 1 Probability Theory: Introduction Basic Probability General In a probability space (Ω, Σ, P), the set Ω is the set of all possible outcomes of a probability experiment. Mathematically, Ω is just
More informationProblem set 1, Real Analysis I, Spring, 2015.
Problem set 1, Real Analysis I, Spring, 015. (1) Let f n : D R be a sequence of functions with domain D R n. Recall that f n f uniformly if and only if for all ɛ > 0, there is an N = N(ɛ) so that if n
More informationNonamenable Products are not Treeable
Version of 30 July 1999 Nonamenable Products are not Treeable by Robin Pemantle and Yuval Peres Abstract. Let X and Y be infinite graphs, such that the automorphism group of X is nonamenable, and the automorphism
More informationTheorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
More informationUniversal Algebra for Logics
Universal Algebra for Logics Joanna GRYGIEL University of Czestochowa Poland j.grygiel@ajd.czest.pl 2005 These notes form Lecture Notes of a short course which I will give at 1st School on Universal Logic
More informationThree hours THE UNIVERSITY OF MANCHESTER. 31st May :00 17:00
Three hours MATH41112 THE UNIVERSITY OF MANCHESTER ERGODIC THEORY 31st May 2016 14:00 17:00 Answer FOUR of the FIVE questions. If more than four questions are attempted, then credit will be given for the
More information5 Measure theory II. (or. lim. Prove the proposition. 5. For fixed F A and φ M define the restriction of φ on F by writing.
5 Measure theory II 1. Charges (signed measures). Let (Ω, A) be a σ -algebra. A map φ: A R is called a charge, (or signed measure or σ -additive set function) if φ = φ(a j ) (5.1) A j for any disjoint
More informationTAIL-FIELDS OF PRODUCTS OF RANDOM VARIABLES AND ERGODIC EQUIVALENCE RELATIONS
TAIL-FIELDS OF PRODUCTS OF RANDOM VARIABLES AND ERGODIC EQUIVALENCE RELATIONS KLAUS SCHMIDT Abstract. We prove the following result: Let G be a countable discrete group with finite conjugacy classes, and
More information(2) E M = E C = X\E M
10 RICHARD B. MELROSE 2. Measures and σ-algebras An outer measure such as µ is a rather crude object since, even if the A i are disjoint, there is generally strict inequality in (1.14). It turns out to
More informationMath 4121 Spring 2012 Weaver. Measure Theory. 1. σ-algebras
Math 4121 Spring 2012 Weaver Measure Theory 1. σ-algebras A measure is a function which gauges the size of subsets of a given set. In general we do not ask that a measure evaluate the size of every subset,
More informationRigorous approximation of invariant measures for IFS Joint work April 8, with 2016 S. 1 Galat / 21
Rigorous approximation of invariant measures for IFS Joint work with S. Galatolo e I. Nisoli Maurizio Monge maurizio.monge@im.ufrj.br Universidade Federal do Rio de Janeiro April 8, 2016 Rigorous approximation
More informationJónsson posets and unary Jónsson algebras
Jónsson posets and unary Jónsson algebras Keith A. Kearnes and Greg Oman Abstract. We show that if P is an infinite poset whose proper order ideals have cardinality strictly less than P, and κ is a cardinal
More informationn [ F (b j ) F (a j ) ], n j=1(a j, b j ] E (4.1)
1.4. CONSTRUCTION OF LEBESGUE-STIELTJES MEASURES In this section we shall put to use the Carathéodory-Hahn theory, in order to construct measures with certain desirable properties first on the real line
More informationA note on transitive topological Markov chains of given entropy and period
A note on transitive topological Markov chains of given entropy and period Jérôme Buzzi and Sylvie Ruette November, 205 Abstract We show that, for every positive real number h and every positive integer
More informationLebesgue measure and integration
Chapter 4 Lebesgue measure and integration If you look back at what you have learned in your earlier mathematics courses, you will definitely recall a lot about area and volume from the simple formulas
More informationEntropy for zero-temperature limits of Gibbs-equilibrium states for countable-alphabet subshifts of finite type
Entropy for zero-temperature limits of Gibbs-equilibrium states for countable-alphabet subshifts of finite type I. D. Morris August 22, 2006 Abstract Let Σ A be a finitely primitive subshift of finite
More information