Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 1. Random Compositions. Evangelos Kranakis School of Computer Science Carleton University

Transcription

1 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 1 Random Compositions Evangelos Kranakis School of Computer Science Carleton University

2 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 2 Outline 1. Random functions and iterations. 2. Iterated random compositions. 3. Markov chain for random compositions. 4. Waiting time until absorption: (a) Lower bound. (b) Upper bound. 5. Open Problems.

3 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 3 Random Functions For n integer, [n] = {1, 2,..., n}. F(m, n) is the space of functions f : [m] [n]. F(n) := F(n, n). A function f is chosen from F(n) randomly with the uniform distribution. Range of f is defined by Range(f) := {y : x [n]f(x) = y}. Question: What is the expected size of the range of a random function?

4 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 4 Values of Random Functions For random function f : [m] [n] and given x [m], y [n], Pr[f(x) = y] = nm 1 n m = 1 n. For random function f : [m] [n] and given y [n], the size of the inverse image f 1 ({y}) satisfies, Pr[ f 1 ({y}) = k] = ( ) m (n 1) m k k n m, for k = 0, 1,..., n.

5 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 5 Graph of a Random Function Given f F(n) the graph G(f) has [n] as its set of vertices and directed edges (x, y) E f(x) = y. Define the k-the iterate f k of f by f 0 = identity function and f k := f f k 1. The orbit (or cycle) of x is defined by O f (x) := {x, f(x), f(f(x)),..., f n 1 (x)}. If L n is the r.v. that counts the length of an orbit it can be shown (Sachkov 1997) Pr[L n = j] = E[L n ] = n k=j n j j=1 (n) k, j = 1,..., n nk+1 n k=j (n) k n k+1

6 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 6 Connected Components of a Random Function The connected components of the graph G(f) consist of the orbits and trees attached to them. f(f(f(x))) f(f(x)) x f(x) The elements of a cycle are called cyclic elements of G(f). The distance of an element from its cycle is called its height.

7 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 7 # of Connected Components of a Random Function Let K n be the r.v. that counts the number of connected components of a random function in F(n). A result of Stepanov (1966) states that Pr[K n = j] = n k=j ( ) n 1 S(k, j), j = 1, 2,..., n k 1 n k E[K n ] = 1 2 V ar[k n ] = 1 2 log n(1 + o(1)) log n(1 + o(1)), where S(k, j) is the # of ways to partition a k element set into j disjoint subsets, a Stirling number of the 2nd kind.

8 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 8 # of Cyclic Elements of a Random Function Let Z n be the r.v. that counts the number of distinct cyclic elements of a random function in F(n). A result of Harris (1973) states that Pr[Z n = k] = k(n) k, k = 0, 1,..., n nk+1 πn E[Z n ] = (1 + o(1)) 2 ( V ar[z n ] = 2 π ) n(1 + o(1)), 2 where (n) k = n(n 1) (n k + 1).

9 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 9 Expected Size of Range of a Random Function f : [m] [n] Consider the indicator function I i : I i = 1 if i Range(f), and I i = 0, otherwise. Then we have [ n ] E[ Range(f) ] = E I i = i=1 n E[I i ] i=1 = ne[i 1 ] = n Pr[I 1 = 1] = n(1 Pr[1 Range(f)]) ( ( = n ) m ). n

10 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 10 Random Functions Cause Shrinkage! Given a random function f F(n), ( E[ Range(f) ] = n n 1 n) 1 n n(1 1/e). [n] [n] f Range(f) n n(e 1)/e

11 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 11 What Causes Shrinkage? Consider a random function f : [m] [n] (with m n): shrinkage is caused by collisions among the elements f(1), f(2),..., f(m), i.e., f(x) = f(y), for some x y. Pr[collision] = Pr[ x y(f(x) = f(y))] = 1 Pr[ x y(f(x) f(y))] = 1 = 1 m 1 i=0 m 1 i=0 n i n ( 1 i ) n (Birthday paradox) Hence, the bigger the m the higher the probability of a collision! We see later that this causes the Markov Chain to skip large states!

12 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 12 Variance of Range(f) for a Random Function f : [m] [n] Let X be the r.v. that counts the size of Range(f) and U = n X. Consider the indicator function I i : I i = 1 if i Range(f), and = 1, otherwise. Observe that I i V ar(x) = V ar(u) = E[U 2 ] E[U] 2 = i j E[I ii j] + i E[I i] E[U] 2 = n(n 1) = n 2 ( ( 1 2 n) m +n ( 1 n) 2 m ( + n 1 1 ) m ( n ) 2m n n ( 1 1 ) ) 2m n (( 1 1 n) m ( 1 2 ) m ). n

13 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 13 Compositions of Random Functions k functions f 1, f 2,..., f k are chosen from F(n) randomly and independently with the uniform distribution. Let f (k) := f k f k 1 f 1. Convention: f (0) = identity function on [n]. f (k) is called a random composition. Question: Since the expected size of the range of a random function is a constant fraction of the size of its domain, how long does it take for a random composition to become constant?

14 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 14 Iterations of Random Functions Model proposed by (Diaconis & Freedman 1999) There is a state space S. There is a family of functions F such that each F F maps the state space into itself F : S S. There is a probability distribution µ on F. If the chain is at state s S it moves to state F (s) by choosing F F at random. The process starts with F 0 and inductively defines X t+1 = F (X t ) where F is a random function F F.

15 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 15 Example I: Linear Affine Functions The state space S is the real line. F = {F +, F } has just two functions defined as follows where 0 < a < 1. F + : R R : x F + (x) = ax + 1 F : R R : x F (x) = ax 1, The probability distribution µ on F is µ(f + ) = µ(f ) = 1/2. Let ξ i = ±1 with probability 1/2, respectively. The process starts with ξ 0 and inductively defines X t+1 = F (X t ) where F is a random function F F. Clearly, X t+1 = ax t + ξ t and the stationary distribution X = ξ 1 + aξ 2 + a 2 ξ 3 + converges since a < 1.

16 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 16 Example II: d-dimensional Affine Functions The state space S is the d-dimensional space R d. F contains a set of functions of the form F : R d R d : x F (x) = Ax + B, A is an d d matrix and B is a d 1 vector. F can be identified with a set of pairs (A, B) of matrices and we have a probability distribution µ on F. The basic chain is X t+1 = A t X t + B t, where A t is an d d matrix and B t a d 1 vector, and (A t, B t ) are idependent and identically distributed. This has applications in fractal geometry.

17 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 17 Example III: Random Compositions States specify the size of the range of a random composition and these states form the state space S. A family of functions { f : f F(n)} maps the state space into itself as follows: Given a function g F(n) already in state s, s f (s) := size of range of f g. The probability distribution on { f : f F(n)} is uniform. If the chain is at state s S it moves to state f (s) by choosing f at random. The process starts with f 0 (identity function) and inductively defines X t+1 = f (X t ) where f is a random function f F(n).

18 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 18 Example IV: Random Compositions of Hashes States specify the size of the range of a random composition and these states form the state space S. H(n) is the set of functions f : [n] [n/2]. A family of functions { h : h H(n)} maps the state space into itself as follows: Given a function g H(n) already in state s, s h (s) := size of range of h g. The probability distribution on { h : h H(n)} is uniform. If the chain is at state s S it moves to state f (s) by choosing f at random. The process starts with a given function h 0 and inductively defines X t+1 = h (X t ) where h is a random function h H(n).

19 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 19 Waiting Time until Absorption For t > 0, we are in state s r iff Range(f (t) ) = r. τ r = {t : Range(f (t) ) = r} is the amount of time in state s r. State s r is visited if τ r > 0. and T is the set of states that are actually visited. Let T be the smallest t for which f (t) is constant, i.e., T = n r=1 τ r s s s n s n 1 s n 2 i 1

20 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 20 How is T Computed The Markov chain starts with the identity function f (0) at time 0 in state s n. By the nature of the problem, states are visited in non-increasing order. It is possible that states may be skipped. Eventually it reaches s 1, the absorbing state. T is really the time it takes to reach the absorbing state s 1. The main result is the following Theorem: E[T ] = 2n(1 + o(1)), as n.

21 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 21 Transition Probabilities For j i, what is the probability f (t) s j given that f (t 1) s i? Given that f (t 1) s i, how many functions f are there such that f f (t 1) has j elements in its range? The are ( n j) ways to choose the range of f f (t 1), S(i, j)j! ways to map the i-element range of f (t 1) onto a given j element set, where S(i, j) is the # of ways to partition a i element set into j disjoint subsets, a Stirling number of the 2nd kind, and n n i ways to map them into [n]. It follows that p(i, j) := Pr[f (t) s j f (t 1) s i ] = ( ) n S(i, j)j!, j n i

22 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 22 Upper Bound on Stirling Numbers S(i, j) S(i, j) = # of ways to partition a i element set into j subsets. Prove by induction S(i, j) (2j) i. For i = 1: S(1, j) 2j We have that S(i, j) = S(i 1, j 1) + js(i 1, j) (Identity) (2(j 1)) i 1 + j(2j) i 1 (Induction) ( (j 1) = (2j) i i 1 2j i + 1 ) 2 (2j) i.

23 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 23 Eigenvalues of the Transition Matrix The transition matrix P := (p(i, j) i,j is lower diagonal. Eigenvalues are the diagonal elements of the matrix, i.e., λ r = p(r, r) ( ) n S(r, r)r! = r n r = r 1 i=1 = 1 ( 1 i n ) ( r ( ) 2) r 4 n + O. n 2 Note: 1 > 1 1 n = λ 2 λ n > 0 and 1 1 λ r+d n 1

24 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 24 Transition Probabilities for Affine Matrices Consider d d matrices over, say, the finite field Z p and let A (0) := I be the identity matrix. A (t) = A t A (t 1), where A t is a random matrix. For t > 0 we are in state r iff rank(a (t) ) = r. τ r = {t : rank(a (t) ) = r} is the amount of time in state s r. Open Question: For j i, compute the transition probabilities p(i, j) := Pr[rank(A (t) ) = j rank(a (t 1) ) = i]. This is equivalent to computing the number of matrices B such that rank(ba) = j, given that rank(a) = i.

25 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 25 Lower Bound on E[T ] We have the identity [ n ] E[T ] = E τ r = = It remains r=2 n E [τ r ] r=2 n E [τ r τ r > 0] Pr[τ r > 0]. r=2 to compute E [τ r τ r > 0], and give a lower bound on Pr[τ r > 0].

26 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 26 Computing E [τ r τ r > 0] This is the expected amount of time you stay in state τ r, given that you visit it? Given τ r > 0, τ r follows the geometric distribution, with probability of success p(r, r) = λ r. E [τ r τ r > 0] = = t=1 1 1 λ r = n ( r 2) tλ t 1 r (1 λ r ) ( 1 + O ( )) r 2 n

27 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 27 Estimating Pr[τ r = 0] We give an upper bound on Pr[τ r = 0]. If τ r = 0 then the state s r is never visited. Therefore there must exist a transition for some positive integers d, j. s r+d s r j, In fact, before moving to state s r j it may stay in state s r+d a number of times t = 0, 1, 2, 3,.... Therefore we must take into account how long we stay in state s r+d given that this state is visited.

28 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 28 We can show that Pr[τ r = 0] = = n r d=1 n r d=1 n r d=1 Upper Bound on Pr[τ r = 0] r 1 j=1 r 1 j=1 r 1 j=1 (n 1) Pr[τ r+d > 0] Pr[τ r+d > 0] p(r + d, r j)p(r + d, r + d) t t=0 p(r + d, r j) 1 λ r+d ( n ) S(r + d, r j)(r j)! r j n r+d (1 λ r+d ) n r d=1 1 n d r 1 j=1 S(r + d, r j) n j Recall: 1) Pr[τ r+d > 0] 1, 2) 1 > 1 1 n = λ 2 λ n > 0 and 1 1 λ r+d n 1

29 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 29 Lower Bound on Pr[τ r > 0] Hence we obtain for r log log n Pr[τ r > 0] 1 (n 1) 1 (n 1) n r d=1 n r d=1 n r 1 n d 1 n d r 1 j=1 r 1 j=1 1 r(2r) r+d 1 (n 1) n d n d=1 ( ) (2l) l+2 1 O n = 1 o(1). S(r + d, r j) n j (2(r j)) r+d n j

30 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 30 Proving the Lower Bound: E[T ] 2n(1 + o(1)) We have the idequality E[T ] l E [τ r τ r > 0] Pr[τ r > 0] r=1 l r=2 = 2n n ( r 2) l r=2 1 r(r 1) l ( 1 = 2n r 1 1 ) r r=2 ( = 2n 1 1 ) l This completes the proof of the lower bound.

31 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 31 Idea of Proof of Upper Bound Recall that T = n m=2 τ m. We split E[T ] into three sums n E[T ] = E [τ m τ m > 0] Pr[τ m > 0] whereby = m=2 ξ 1 m=2 + ξ 1 = ξ 2 = ξ 2 m=ξ n log n n log 2 n n m=ξ 2 +1 and make upper bound estimates on each of them.

32 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 32 Observe that 1st Sum: = ξ 1 m=2 E [τ m τ m > 0] Pr[τ m > 0] 1st Sum = = ξ 1 m=2 ξ 1 m=2 ξ 1 m=2 2n 1 1 λ m 1 ( m 2 ) n + O(m4 /n 2 ) ( m 2 ξ 1 m=2 n ) + O(m4 /n) 1 m(m 1) = 2n(1 1/ξ 1 ) = 2n(1 + o(1))

33 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 33 2nd Sum: = ξ 2 m=ξ 1 +1 E [τ m τ m > 0] Pr[τ m > 0] Observe that λ ξ1 = log n + O(1/ log2 n). 2nd Sum = ξ 2 m=ξ λ ξ1 1 1 λ m = O(ξ 2 log n) = O(n/ log n) = o(1). ξ 2 m=ξ

34 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 34 3rd Sum: = n m=ξ 2 +1 E [τ m τ m > 0] Pr[τ m > 0] Observe that max m>ξ2 1 1 λ m = 1 1 λ ξ2 +1. Hence, 3rd Sum n 1 Pr[τ m > 0] 1 λ m m=ξ n Pr[τ m > 0] 1 λ ξ2 +1 m=ξ n 1 exp ( ( ξ 22 ) ) Pr[τ m > 0] /n n 2 Pr[τ m > 0], m=ξ 2 +1 m=ξ 2 +1 for n large enough. It remains to bound the RHS above.

35 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 35 On Skipping Large States To deal with n m=ξ 2 +1 Pr[τ m > 0] we will show that every hit (i.e., τ m > 0) is followed by many (i.e., β) misses (i.e., τ m δ = 0, forall δ such that 1 δ β) Let us define β := β(n) = 1 2 ( ξ 2 n + n ( 1 1 ) ) ξ2 n Observe that β(n) >> n log 4 n. Suppose we are in state s m at time t 1, i.e., Range(f (t 1) ) = m, and select the next function f t at random.

36 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 36 Claim 1: If B > β then τ m δ = 0, for 1 δ β. Let h be the restriction of f t to the range of f (t 1). Let R be the cardinality of the range of h, and B = m R. To prove the claim notice that B > β m R > β R < m β and Range(f (t) ) = f (t) ([n]) = f t (f (t 1) ([n])) = h(range(f (t 1) )) = R.

37 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 37 Observe that Claim 2: E[B] ξ 2 n + n(1 1 n )ξ 2 >> n log 4 n. E[B] = E[m R] = m E[R] = m n + n ( 1 1 ) m n > ξ 2 n + n(1 1 n )ξ 2 = 2β n >> log 4 n. Also ( recall, V ar(b) = (1 ) n 2 2 m ( ) ) n 1 1 2m n + n (( 1 1 n) m ( 1 2 n) m ) = O(m).

38 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 38 Claim 4: Pr[ 1 δ β(τ m δ = 0) τ m > 0] By Chebeshev s Inequality and since m > ξ 2 we have Pr[ B β] Pr [ B 12 ] E[B] 4V ar(b) (E[B]) 2 ( m log 8 n = O = o(1). Hence, Pr[ B > β] = 1 o(1). It follows, by Claim 1, n 2 Pr[ 1 δ β(τ m δ = 0) τ m > 0] = 1 o(1). )

39 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 39 Back to the 3rd Sum: = n m=ξ 2 +1 E [τ m τ m > 0] Pr[τ m > 0] Define χ m = 1 if τ m > 0 and χ m = 0, otherwise. Recall that n 3rd Sum 2 Pr[τ m > 0] = 2 = 2E m=ξ 2 +1 n m=ξ 2 +1 = 2E[V ], n E[χ m ] χ m m=ξ 2 +1 where V := n m=ξ 2 +1 χ m. Define W := n m=ξ 2 +1 (1 χ m) and observe that V + W = n ξ 2.

40 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 40 Back to the 3rd Sum: = n m=ξ 2 +1 E [τ m τ m > 0] Pr[τ m > 0] Note that if τ m > 0 and 1 δ β(τ m δ = 0) then these β missed states contribute exactly β to W. Hence, if we define J m = χ m β δ=1 (1 χ m δ) then W β m>ξ 2 J m. Hence, E[W ] β m>ξ 2 E[J m ] = β m>ξ 2 Pr[J m = 1] = β m>ξ 2 Pr[τ m > 0] Pr[ 1 δ β(τ m δ = 0) τ m > 0] β(1 + o(1)) m>ξ 2 Pr[τ m > 0] = β(1 + o(1))e[v ].

41 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 41 Back to the 3rd Sum: = n m=ξ 2 +1 E [τ m τ m > 0] Pr[τ m > 0] It follows that Hence, E[V ] = n ξ 2 E[W ]) n ξ 2 β(1 + o(1))e[v ]) 3rd Sum 2E[V ] 2(n ξ 2 ) 1 + β(1 + o(1)) = O(log 4 n) = o(n). This completes the proof of the theorem.

42 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 42 Another Idea: Hitting Times For r = 1, 2,..., n, let t r be the (hitting) time that the chain spends in transient state r until absorbtion by state 1. Let the (n 1) (n 1) matrix Q be obtained from the transition matrix P by removing the first row and column. It is easy to see that t 1 = 1 and t r = 1 r=1 + r r =2 p(r, r )t r If I is the unit matrix and t is the vector of hitting times then t = I + Qt, which is equivalent to t(i Q) = I. So to compute the hitting times it is enough to compute the inverse of the matrix I Q. Easy to prove: if lim k Q k = 0 then (I Q) 1 = k=0 Qk. Speed of convergence depends on 2nd largest eigenvalue!

43 Dalal-Schmutz (2002) and Diaconis-Freedman (1999) 43 Open Questions Consider the space H(n) := F(n, n/2) of hash functions. Consider different probability distributions on F(n). The reason is that in practice one has preference over certain types of random functions. Consider a space {(A, B)} of pairs of d d (random) matrices and the functions x Ax + B. States are determined by the rank of a random matrix. This problem is equivalent to estimating the time T until the product of T random matrices is 0.