Coin tossing space Think of a coin toss as a random choice from the two element set {0,1}. Thus the set {0,1} n represents the set of possible outcomes of n coin tosses, and Ω := {0,1} N, consisting of all sequences (t n ) n N, represents the set of possible outcomes of tossing a coin infinitely many times. If the coin is unbiased and the tosses are independent, each of the 2 n outcomes in {0,1} n is equally probable. So {0,1} n is equipped with a probability measure µ n, which is 2 n times counting measure on {0,1} n. (All subsets of {0,1} n are measurable.)
Compatible probability measures Assume that m < n. Tossing a coin n times, and then ignoring all but the first m of them, is represented by a projection map π mn : {0,1} n {0,1} m : π mn (t 1,..., t n ) = (t 1,..., t m ). An event depending on (t 1,..., t m ) is a set E {0,1} m. It has probability µ m (E). The same event considered as a subset of {0,1} n is π 1 mn (E). It does in fact have the same probability, in {0,1} n, as E does, in {0,1} m : µ n ( π 1 mn (E)) = µ m (E), E {0,1} m. (1) Think of this as a compatibility result: All the different probability measures µ n give the same result for those events they can measure. (Exercise: Show this. Hint: Show that for each s E there are 2 n m distinct sequences t π 1 mn (E) with π mn(t) = s.)
Lots and lots of projections All these projection maps π nm are compatible in the sense that π km π mn = π kn, k < m < n. It is helpful to picture the finite coin tossing spaces as an infinite sequence: {0,1} {0,1} 2 {0,1} 3 {0,1} n {0,1} n+1 where the arrows shown are π 12, π 23,..., π n,n+1. At the far right we can put the infinite coin tossing space Ω. (Category theorists call that the projective limit of the above diagram, but never mind that.) There is also a projection map π n : Ω {0,1} n, given by π n (t 1, t 2,...) = (t 1,..., t n ). Clearly, the compatibility of projections extends to this one: π mn π n = π m, m < n. (2)
Finitely determined events and their probabilities Some events in Ω depend only on a finite number of coin tosses. (By this I mean that to determine whether t Ω belongs to some event E, you only need to look at t 1,..., t n for some n, and that n is independent of t.) If E {0,1} n then π 1 n (E) = {t Ω: (t 1,..., t n ) E} is such an event, and I expect you can convince yourself that every such finitely determined event has this form. (If not, consider that a definition.) Define A n = {π 1 n (E): E {0,1}n }, n N. Then A n is a σ-algebra of subsets of Ω for each n N, and moreover A m A n when m < n (for the equality π mn π n = π m implies π 1 m (E) = π 1 n (π 1 nm (E)) when E {0,1} m ). Let C = A n. n N It is not hard to show that, since (A n ) is an increasing sequence of algebras, the union C is also an algebra of sets (do it!). We can define a function ι on C by ι ( π 1 n (E)) = µ n (E), E {0,1} n. (Exercise: Show that this is well defined. Hint: Use (1) and (2).)
Extending the measure The function ι represents the probability of any finitely determined event. But what about other events? How can we discuss the probability that 1 n lim t n n k = 1 2, for example? The set of t Ω satisfying this equality is clearly not finitely determined. What is needed is to extend ι to a measure. It is defined on the algebra C, which is not a σ-algebra. But C is a semialgebra, so in order to show that we can extend ι to a measure, we need to show that ι( ) = 0, that ι is finitely additive, and the appropriate subadditivity condition. For the first property, ι( ) = µ n (π 1 n ( )) = µ n( ) = 0. For the second, if F 1,...,F m C are pairwise disjoint then there is some n so that F k A n for k = 1,...,m. Write F k = π 1 n (E k). Then the sets E k must be pairwise disjoint, and ( m ) ( m ) m m ι F k = µ n E k = µ n (E k ) = ι(f k ). Finally, assume that F,F k C with F k N F k. I claim that a finite number of the sets F k will cover F, i.e., F F 1 F n for some n. If this is true, then ( n ) ι(f ) ι F k n ι(f k ) ι(f k ), which is the subadditivity condition we need to show. To prove the claim, assume instead it is false. That is, we assume that each set A n, defined by A n = F \(F 1 F n ), is nonempty. Note that A n A, and A 1 A 2 A 3. We now define a sequence (u k ) in {0,1} by induction, with the property that A n has some member t with (t 1,..., t k ) = (u 1,...,u k ) infinitely often (meaning for infinitely many n), pick u 1 = 0. First, if A n has some member t with t 1 = 0 infinitely often, let u 1 = 0. Otherwise, A n must have some member t with t 1 = 1 infinitely often. Accordingly, pick u 1 = 1. The induction step is similar; assume u 1,...,u k have been chosen, consider only t A n with (t 1,..., t k ) = (u 1,...,u k ) (which exist infinitely often), and choose u k+1 = 0 or u k+1 = 1 so that (t 1,..., t k, t k+1 ) = (u 1,...,u k,u k+1 ) infinitely often. For any n, since A n A there is some k with A n A k. By construction, A m has a member t with π k (t) = π k (u) for infinitely many m. In particular, this is so for some m n. But then A m A n, so there is some t A n with π k (t) = π k (u). Since A n A k, it follows that u A n. In other words, u n N A n, but that means that F n N F n. This is a contradiction, which proves the claim.
Probability on the coin tossing space From the general theory, we now have a measure µ on a σ-algebra A extending C, and hence all the algebras A n. Moreover, by construction µ ( π 1 n (E)) = µ n (E) for all E {0,1} n, so our probability measure certain captures our intuition about the distribution of n coin tosses. The nth toss is represented by a function T n : Ω {0,1}, defined by T n (t) = t n. T n is a stochastic variable, i.e., a measurable function on Ω. It is not difficult to show that the tosses T n are stochastically independent: If G 1,...,G n R are Borel sets, then ( n ) µ T 1 k (G k) = n µ ( T 1 k (G k) ). (Note that T 1 k (G k) is one of four sets:, {t Ω: t k = 0}, {t Ω: t k = 1}, or Ω.)