Applications of Discrete Mathematics to the Analysis of Algorithms

Applications of Discrete Mathematics to the Analysis of Algorithms Conrado Martínez Univ. Politècnica de Catalunya, Spain May 2007

Goal Given some algorithm taking inputs from some set Á, we would like to analyze the performance of the algorithm as a function of the input size (and possibly other parameters).

Why? To predict the resources (time, space.... ) that the algorithm will consume To compare algorithm with competing alternatives To improve the algorithm, by spotting the performance bottlenecks To explain observed behavior

The performance Á Æ depends on each particular instance of the input We have to introduce some notion of size: Á Æ; we may safely assume that each Á Ò Ü ¾ Á Ü Ò is finite Worst case: [worst] Òµ ÑÜ Üµ Ü ¾ Á Ò

To analyze typical behavior or the performance of randomized algorithms, we have to assume some probabilistic distribution on the input and/or the algorithm s choices; hence, we consider the performance as a family of random variables Ò Ò¼ ; Ò Á Ò Æ Average case: [avg] Òµ Ò ¼ È Ò

When we assume uniformly distributed inputs ÈÜ ½ Á Ò for all Ü ¾ Á Ò our problem is one of counting, e.g., È Ü¾ÁÒ Ò Üµ Á Ò

One of the most important tools in the analysis of algorithms are generating functions: Þ Ùµ Ò¼ ¼ È Ò Þ Ò Ù For the uniform distribution Þ Ùµ È Ò¼ È¼ ÒÞ Ò Ù È Þ Ùµ Ò¼ ÒÞ Ò Þ ½µ with Ò Ü ¾ Á Ü Ò Üµ and Ò Á Ò

The equations before can be expressed symbolically Þ Ùµ Ü¾Á Þ Ü Ù Üµ The ratio of the Ò th coefficients of Þ Ùµ and Þ ½µ is the probability generating function of Ò Ô Ò Ùµ ¼ È Ò Ù ÞÒ Þ Ùµ Þ Ò Þ ½µ

Taking derivatives w.r.t. Ù and setting Ù ½ we get the expected value, second factorial moment,... Öµ Þµ Ö Þ Ùµ Ù Ö Ò¼ Ù½ Ò Ö Þ Ò For example, Î Ò Ò ¾ Ò ¾ Þ Ò ¾µ Þµ Þ Ò Þµµ ¾

The symbolic method translates combinatorial constructions to functional equations over generating functions. Example: Consider the counting generating function of a combinatorial class : If then Þµ «¾ Þ «Þµ Þµ Þµ Ò¼ µ¾ Ò Þ Ò Þ «¾ Þ «¼ ¾ ½ Þ ¼ ¾ Þ ½

A dictionary of (labelled) combinatorial constructions and G.F. s ËÕ µ ½ Þ ½ ½ ËØ µ ÜÔ µ ÝÐ µ ÐÓ ½ ½

A trivial example: since a binary tree is either an empty tree (leaf) or a root together with two binary (sub)trees, we have Hence the counting GF of binary trees is Þµ ½ Þ ¾ Þµ

Solving the equation before for Þµ and since ¼µ ¼ ½, Þµ Ô ½ ½ Þ ¾Þ ½ Þ ¼ Þ ¼ Extracting the Ò th coefficient of Þµ we find ¾Ò Þ Ò Ò Þµ Ò ½

A more sophisticated example arises in the analysis of the number of branch mispredictions made by quicksort (Kaligosi, Martínez, Sanders, 2006).

We have a random permutation of ½Ò and we scan it from left to right. Assume ½. For ¾, we say there is a left brach misprediction whenever ½ and, or ½ and. For Ò, there is a right BM if ½ and, or ½ and. Additionally, there is a left BM if ¾ and a right BM if Ò.

We want to know the expected number of branch mispredictions when the choosen pivot is the th element of the array. We transform the original problem to counting bitstrings. Given a bistring Ü of length starting with a 0 and containing Ø ½ 0 s, the number of left BMs is the number of times we find a 0 followed by a 1, or a 1 followed by a 0 in Ü. We go from a permutation to a bitstring Ü by setting Ü ¼ if ½, and Ü ½ otherwise.

0 A 1 1 B 0 ÒØÖ number of strings of length Ò with Ø 0 s ending at state and incurring Ö BMs ÒØÖ number of strings of length Ò with Ø 0 s ending at state and incurring Ö BMs

0 A 1 1 B 0 Ü Ù Þµ Ü Ù Þµ ÒÖØ ÒÖØ ÒØÖ Ü Ø Ù Ö Þ Ò ÒØÖ Ü Ø Ù Ö Þ Ò

0 A 1 1 B 0 ½ ÜÞ ÜÙÞ Þ ÞÙ

0 A 1 1 B 0 ½ Þ ÙÞ ÜÙ ½ Þµ ½ ÙÝ ¾ Þ ¾ ½ Þ Take derivatives with respect to Ù, set Ù ½, and extract the coefficient Ê Ø of Þ ½ Ý Ø in ; then multiply by the number of random permutations producing a bitstring with Ø ½ 0 s among the first bits and sum for all Ø.

Another important set of techniques comes from complex variable analysis. Under suitable technical conditions, if Þµ is analytic in a disk Þ ¾ Þ ½ and has a single dominant singularity at Þ ½ then Þµ Þµ µ Þ Ò Þµ Þ Ò Þµ This is one of the useful transfer lemmas of Flajolet and Odlyzko (1990). Many other similar results are extremely useful when computing asymptotic estimates for the Ò th coefficient of a generating function.

In recent years, complex analysis techniques and perturbation theory have been used to prove powerful results such as Hwang s quasi power theorem, which allows one to prove the convergence in law to a Gaussian distribution of many combinatorial parameters in strings, permutations, trees, etc., as well as local limits and the speed of convergence.

Another example is motivated by the analysis of a combinatorial algorithm that shuffles two trees. Given two binary trees Ì ½ and Ì ¾ their shuffle or intersection is defined as follows Ì ½ Ì ¾ if Ì½ or Ì ¾, Æ Ä ½ Ä ¾ Ê ½ Ê ¾ µ if Ì Æ Ä Ê µ

Let Ë Ü Ýµ Ì ½Ì ¾µ¾ ÈÖ Ì ½ Ì ¾ µ Ì ½ Ì ¾ Ü Ì½ Ý Ì¾ The coefficient of Ü Ñ Ý Ò in Ë Ü Ýµ is the average size of the intersection of a pair of trees with sizes Ñ and Ò, resp.

If we assume independently drawn trees then ÈÖ Ì ½ Ì ¾ µ ÈÖ Ì ½ µ ÈÖ Ì ¾ µ If we assume that the total size of Ì ½ Ì ¾ µ is Ò and all possible partitions are equally likely (binary search tree probability model) ÈÖ Ì ½ Ì ¾ µ ÈÖ Ì ½µ ÈÖ Ì ¾ µ Ì ½ Ì ¾ ½

Furthermore, In the uniform probability model ÈÖ Ì µ ½ Ò, with Ò Ì. In the binary search tree model ÈÖ Ì µ ½ if Ì, ÈÖ Äµ ÈÖ Êµ Ì if Ì Æ Ä Êµ.

In the binary search tree probability model, the symbolic method yields the hyperbolic PDE that Ë Ü Ýµ satisfies ¾ Ë Ü Ýµ ÜÝ ½ ½ Üµ ¾ ½ Ýµ ¾ Ë Ü Ýµ ¾ ½ Üµ ½ Ýµ and Ë Ü ¼µ Ë ¼ Ýµ ¼.

If we consider Þµ ½ Þ Þ ¼ Ë Ø Øµ Ø then Þ Ò Þµ is the average size of the intersection of a pair of trees of total size Ò drawn according to the binary search tree probability model.

A delicate analysis of the solution of the PDE for Ë Ü Ýµ shows that ½ Ô Ô Þµ ¾ ¾µ Â ¼ ¾ ¾i ÐÒ ½ Þ with Â ¼ Üµ the Bessel function of first kind of order 0.

The function Þµ has a unique dominant singulary at Þ ½; as Þ ½, we have ½ Ô Â ¼ ¾ ¾i ÐÒ ½ Ô ¾ ½ Þ ½ Ö ÐÒ ½ ½ Þ ½ ½ Ç ½ Þµ ¾Ô ¾ ½ ÐÒ ½ Þµ

Applying transfer lemmas Ô Ò Þ Ò Þµ Ò¾ ¾ ½ ½ Ô ½ Ç ÐÒ Ò ÐÓ Ò and ¾ Ô ¾ ¾ Ô ¾ Ô ¾µ ¼¼¼

Conclusion We ve just scratched the surface of the rich number of applications of discrete mathematics for the analysis of algorithms. I hope I have convinced you that Analysis of Algorithms is deeply rooted in mathematics, most notably discrete mathematics.

Several of the talks in the minisymposium show nice applications of the techniques briefly described here, arising in the analysis of sorting and selection algorithms, automata theory, multidimensional data structures, polynomial factorization, decomposable combinatorial structures, union find data structures, etc. Other talks will present other elegant and powerful techniques that we haven t presented now.