3. Generating Functions

Transcription

1 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions

2 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3a.GFs.OGFs

3 Ordinary generating functions Definition. () = is the ordinary generating function (OGF) of the sequence,,,...,,... Notation. [z N ]A(z) is the coefficient of z N in A(z) sequence 1, 1, 1, 1, 1,... 1, 1/2, 1/6, 1/24,... OGF =! = [z N ]e z = 1/N! Significance. Can represent an entire sequence with a single function. 3

4 Operations on OGFs: Scaling If () = is the OGF of,,,...,,... then () = is the OGF of,,,,... sequence 1, 1, 1, 1, 1,... 1, 2, 4, 8, 16, 32,... OGF = = [ ] = 4

5 Operations on OGFs: Addition If () = is the OGF of,,,...,,... and () = is the OGF of,,,...,,... then ()+() is the OGF of +, +, +,..., +... Example: sequence 1, 1, 1, 1, 1,... 1, 2, 4, 8, 16, 32,... 0, 1, 3, 7, 15, 31,... OGF = = 5

6 Operations on OGFs: Differentiation If () = is the OGF of,,,...,,... then () = is the OGF of,,,,...,,... OGF = ( ) = ( ) = ( ) ( ) = ( ) + sequence 1, 1, 1, 1, 1,... 0, 1, 2, 3, 4, 5,... 0, 0, 1, 3, 6, 10,... 0,..., 1, M+1, (M+2)(M+1)/2,... ( ) + = + 1, M+1, (M+2)(M+1)/2,... 6

7 Operations on OGFs: Integration If () = is the OGF of,,,...,,... then () =,,,,..., is the OGF of,... Example: OGF = sequence 1, 1, 1, 1, 1,... ln = 0, 1, 1/2, 1/3, 1/4, 1/5,... 7

8 Operations on OGFs: Partial sum If () = is the OGF of,,,...,,... then () is the OGF of, +, + +,... Proof. Distribute Change n to n k Switch order of summation. Change k to n k () = = + = = ( ) = ( ) 8

9 = = = = Operations on OGFs: Partial sum If () = is the OGF of,,,...,,... then () is the OGF of, +, + +,... Example: OGF () = + sequence = 1, 1, 1, 1, 1,... ln = 0, 1, 1/2, 1/3, 1/4, 1/5,... ln = 1, 1 + 1/2, 1 + 1/2 + 1/3,... 9

10 Operations on OGFs: Convolution If () = is the OGF of,,,...,,... () = and is the OGF of,,,...,,... then ()() is the OGF of, +,...,,... Proof. ()() = Distribute = + Change n to n k = Switch order of summation. = 10

11 Operations on OGFs: Convolution If () = is the OGF of,,,...,,... () = and is the OGF of,,,...,,... then ()() is the OGF of, +,...,,... Example: OGF = sequence 1, 1, 1, 1, 1,... ( ) = ( + ) 1, 2, 3, 4, 5,... 11

12 Expanding a GF (summary) The process of expressing an unknown GF as a power series (finding the coefficients) is known as expanding the GF. Techniques we have been using: 1. Taylor theorem: () =()+ () + ()! + ()! + () +...! Example. = + +! +! +! Reduce to known GFs. Example. [ ] ( ) ln = = Integrate then convolve to get ln with ln 12

13 In-class exercise Exercise 3.4 Prove that =( + )( + ) ln 1. Find GF for LHS (convolve with ) ln ( ) 2. Expand GF to find RHS coefficients (convolve ln with ( ) ) [ ] ln ( ) = ( + ) 3. Do some math =( + ) =( + )( + =( + )( + + ) ) 13

14 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3a.GFs.OGFs

15 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3b.GFs.recurrences

16 Solving recurrences with OGFs General procedure: Make recurrence valid for all n. Multiply both sides of the recurrence by z n and sum on n. Evaluate the sums to derive an equation satisfied by the OGF. Solve the equation to derive an explicit formula for the OGF. (Use the initial conditions!) Expand the OGF to find coefficients. recurrence sequence 16

17 Solving recurrences with GFs For linear recurrences with constant coefficients, the GF equation is a polynomial, so the general procedure is a algorithm. Example 4 from previous lecture. Expand. = = = Make recurrence valid for all n. = + Multiply by z n and sum on n. () =() ()+ Solve. () = Use partial fractions: solution must be of the form Solve for coefficients. () = = = + = Solution is c 0 = 1 and c 1 = 1 () = 17

18 Solving linear recurrences with GFs For linear recurrences with constant coefficients, the GF equation is a polynomial, so the general procedure is a algorithm. Example with multiple roots. = + =, = = Make recurrence valid for all n. = + + Multiply by z n and sum on n. () =() ()+ ()+ Solve. () = + Simplify. () = ( ) ( )( ) = ( ) Expand. = multiplicity 3 gives terms of the form n 2 β n, etc. 18

19 Solving linear recurrences with GFs For linear recurrences with constant coefficients, the GF equation is a polynomial, so the general procedure is a algorithm. Example with complex roots. = + =, = = Make recurrence valid for all n. = + + Multiply by z n and sum on n. () =() ()+ ()+ Solve. Simplify. () = Use partial fractions. () = + ( )( + ) = ( + ) () = + + Expand. = ( +( ) )= ( +( ) ) 1, 0, -1, 0, 1, 0, -1, 0,

20 Solving linear recurrences with GFs (summary) Solution to = is a linear combination of t terms. Suppose the roots of the polynomial are β1, β2,..., βr where the multiplicity of βi is mi so = Solution is < β + < β < β t terms The t constants cij are determined from the initial conditions. Note: complex roots (and 1) introduce periodic behavior. 20

21 Solving the Quicksort recurrence with OGFs = + + = ( + )+ Multiply both sides by N. Multiply by zn and sum. = ( + ) + homogeneous equation Evaluate sums to get an ordinary differential equation ( )= ( ) ( ) + ( )= ( )/( solution (integration factor) ( ) = /( Solve the ODE. (( ) Integrate. Expand. ( )) = ( ) =( ) ( )= =[ ] ( ( ) ) ln ) ( ) ( ( ) ) ) ( ) ( ) = ln = ( + )( + ) 21

22 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3b.GFs.recurrences

23 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3c.GFs.Catalan

24 Catalan numbers How many triangulations of an (N+2)-gon? Tk TN 1 k T0 = 1 T1 = 1 T2 = 2 = < + T3 = 5 T4=14 24

25 Catalan numbers How many gambler s ruin sequences with N wins? Tk TN 1 k T0 = 1 T1 = T2 = T3 = 5 = < + T4=14 25

26 Catalan numbers How many binary trees with N nodes? Tk TN 1 k k nodes N 1 k nodes = < + 26

27 Catalan numbers How many trees with N+1 nodes? Tk TN 1 k k+1 nodes N k nodes = < + 27

28 Solving the Catalan recurrence with GFs Recurrence that holds for all N. = + δ < Multiply by z N and sum. () = < + Switch order of summation () = + > Change N to N+k+1 () = + ++ convolution (backwards) Distribute. () = + () = + () 28

29 Common-sense rule for working with GFs It is always worthwhile to check your math with your computer. Known from initial values: () = Check: () = + () sage: ZP.<z> = ZZ[] sage: 1 + z*(1+z+2*z^2+5*z^3+14*z^4)*(1+z+2*z^2+5*z^3+14*z^4) 196*z^ *z^8 + 81*z^7 + 48*z^6 + 42*z^5 + 14*z^4 + 5*z^3 + 2*z^2 + z + 1 not valid because z^5 and beyond missing in factors 29

30 Solving the Catalan recurrence with GFs (continued) Functional GF equation. () = + () Solve with quadratic formula. () = ( ± ) Expand via binomial theorem. () = ( ) Set coefficients equal = Expand via definition. = Distribute ( 2) N among factors. = + ( )+ ( )( )...( )( ) + ( + )! ( ) ( + )! Substitute (2/1)(4/2)(6/3)... for 2 N. = ( +! = + ) 30

31 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3c.GFs.Catalan

32 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3d.GFs.EGFs

33 Exponential generating functions (EGFs) Definition. () =! is the exponential generating function (EGF) of the sequence,,,...,,... sequence 1, 1, 1, 1, 1,... 1, 2, 4, 8, 16, 32,... 1, 1, 2, 6, 24, EGF! =! =!! = 33

34 Operations on EGFs: Binomial convolution If () =! is the EGF of,,,...,,... and () = is the EGF of,,,...,,...! then ()() is the EGF of, +,...,,... Proof. ()() = Distribute. = Change n to n k = Multiply and divide by n! = Switch order of summation. =!!!! +! ( )!!! 34

35 Solving recurrences with EGFs Choice of EGF vs. OGF is typically dictated naturally from the problem. Example. = Multiply by z n /n! and sum on n. () = Switch order of summation. () =!! Change n to n+k. () = Simplify. () = + (/)!! + ( + )! binomial convolution (backwards) Distribute. () = Evaluate and telescope. (/)! () = (/) = +/+/+/+... =! convergence not assured Expand. = Check. = 35

36 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3d.GFs.EGFs

37 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3e.GFs.counting

38 Counting with generating functions An alternative (combinatorial) view of GFs Define a class of combinatorial obects with associated size function. GF is sum over all members of the class. Example. T set of all binary trees t number of internal nodes in t T TN number of t T with t = N () T = Decompose from definition Distribute () = + ( )( ) = tl tl nodes tr tr nodes = + () 38

39 Combinatorial view of Catalan GF Each term z N in the GF corresponds to an object of size N. Collect all the terms with the same exponent to expose counts. Each term z i z j in a product corresponds to an object of size i + j. () = = () = + () = + ( )( ) =

40 Values of parameters ("costs") are often the object of study in the analysis of algorithms. How many 1 bits in a random bitstring? (Easy) How many leaves in a random binary tree? (Not so easy) leaf 40

41 Computing expected costs by counting An alternative (combinatorial) view of probability Define a class of combinatorial obects. Model: All objects of size N are equally likely P P P = () P = Expected cost of an object of size N = PNk/PN is the probability that the cost of on object of size N is k cumulated cost Def. Cumulated cost is total cost of all objects of a given size. Expected cost is cumulated cost divided by number of objects. 41

42 Counting with generating functions: cumulative costs An alternative (combinatorial) view of GFs Define a class of combinatorial obects. Model: All objects of size N are equally likely GF is sum over all members of the class. P P P = () Counting GF () P = Cumulative cost GF Average cost () () = P [ ]()/[ ]() Bottom line: Reduces computing expectation to GF counting 42

43 Warmup: How many 1 bits in a random bitstring? B is the set of all bitstrings. b is the number of bits in b. ones(b) is the number of 1 bits in b. BN is the # of bitstrings of size N (2 N ). CN is the total number of 1 bits in all bitstrings of size N. Counting GF. () = = = Cumulative cost GF. () = Average # 1 bits in a random bitstring of length N. = () + + ( ) = ()+() = ( ) [ ]() [ ]() = = 0 b 1 b ( ) = () 43

44 Leaves in binary trees are internal nodes whose children are both external. Definitions: TN is the # of binary trees with N nodes. tnk is the # of N-node binary trees with k leaves CN is the average # of leaves in a random N-node binary tree C0 = 0 t11=1 C1 = 1 t21=2 C2 = 1 t31=4 t32=1 C3 = 1.2 t41=8 t42=6 C4 = Q. How many leaves in a random binary tree? 44

45 How many leaves in a random binary tree? T is the set of all binary trees. t is the number of internal nodes in t. leaves(t ) is the number of leaves in t. TN is the # of binary trees of size N (Catalan). CN is the total number of leaves in all binary trees of size N. Counting GF. () = T = = + Cumulative cost GF. () = () Average # leaves in a random N-node binary tree. [ ]() [ ]() = [ ]() Next: Derive a functional equation for the CGF. 45

46 CGF functional equation for leaves in binary trees CGF. () = Decompose from definition. () tl tl nodes leaves(tl) leaves tr tr nodes leaves(tr) leaves () = + ( )+( ) + + = + ( ) + ( ) = + ()() 46

47 How many leaves in a random binary tree? CGF. () = () Decompose from definition. () = + ( )+( ) + + = + ()() Compute number of trees T N. () = () =[ ] Catalan numbers = ( ) = + ( ) Compute cumulated cost C N. () = + ()() = () = =[ ] = Compute average number of leaves. / = + = ( + ) ( ) / 47

48 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions OF OGFs Solving recurrences Catalan numbers EGFs Counting with GFs 3e.GFs.counting

49 Exercise 3.20 Solve a linear recurrence. Initial conditions matter. AN INTRODUCTION TO THE ANALYSIS OF ALGORITHMS S E C O N D E D I T I O N R O B E R T S E D G E W I C K PHILIPPE FLAJOLET 49

50 Exercise 3.28 The art of expanding GFs. AN INTRODUCTION TO THE ANALYSIS OF ALGORITHMS S E C O N D E D I T I O N R O B E R T S E D G E W I C K PHILIPPE FLAJOLET 50

51 Assignments for next lecture 1. Use a symbolic mathematics system to check initial values for C(z) = z + 2C(z)T(z). 2. Read pages in text. AN INTRODUCTION TO THE ANALYSIS OF ALGORITHMS S E C O N D E D I T I O N 3. Write up solutions to Exercises 3.20 and R O B E R T S E D G E W I C K PHILIPPE FLAJOLET 51

52 ANALYTIC COMBINATORICS P A R T O N E 3. Generating Functions