Copyright 2000, Kevin Wayne 1

Transcription

1 CS 580: Algorithm Design and Analysis.8 Knapsack Problem Jeremiah Blocki Purdue University Spring 08 Reminder: Homework 6 has been released. Weighted Vertex Cover Polynomial Time Approximation Scheme Theorem. -approximation algorithm for weighted vertex cover via LP rounding. PTAS. ( + )-approximation algorithm for any constant > 0. Load balancing. [Hochbaum-Shmoys 987] Euclidean TSP. [Arora 996] Theorem. [Dinur-Safra 00] If P NP, then no -approximation for <.3607, even with unit weights. Consequence. PTAS produces arbitrarily high quality solution, but trades off accuracy for time Open research problem. Close the gap. This section. PTAS for knapsack problem via rounding and scaling. Unique Games Conjecture: Implies there is no -approximation for <.99999, even with unit weights Disagreement among about validity of this conjecture 4 Copyright 000, Kevin Wayne

2 Knapsack Problem Knapsack Problem: Dynamic Programming Knapsack problem. we'll assume w i W Given n objects and a "knapsack." Item i has value v > 0 and weighs w i i > 0. Knapsack can carry weight up to W. Goal: fill knapsack so as to maximize total value. Def. OPT(i, w) = max value subset of items,..., i with weight limit w. Case : OPT does not select item i. OPT selects best of,, i using up to weight limit w Case : OPT selects item i. new weight limit = w w i Ex: { 3, 4 } has value 40. W = Item Value Weight OPT selects best of,, i using up to weight limit w w i 0 if i 0 OPT(i, w) OPT(i, w) if w i w maxopt(i, w), v i OPT(i, w w i ) otherwise Running time. O(n W). W = weight limit. Not polynomial in input size! 5 7 Knapsack is NP-Complete Knapsack Problem: Dynamic Programming II 6 KNAPSACK: Given a finite set X, nonnegative weights w i, nonnegative values v i, a weight limit W, and a target value V, is there a subset S X such that: w i W is v i V is SUBSET-SUM: Given a finite set X, nonnegative values u i, and an integer U, is there a subset S X whose elements sum to exactly U? Claim. SUBSET-SUM P KNAPSACK. Pf. Given instance (u,, u n, U) of SUBSET-SUM, create KNAPSACK instance: v i w i u i V W U u i is u i is U U Def. OPT(i, v) = min weight subset of items,, i that yields value exactly v. Case : OPT does not select item i. OPT selects best of,, i- that achieves exactly value v Case : OPT selects item i. 8 consumes weight w i, new value needed = v v i OPT selects best of,, i- that achieves exactly value v 0 if v 0 if i 0, v > 0 OPT(i, v) OPT(i, v) if v i v minopt(i, v), w i OPT(i, v v i ) otherwise Running time. O(n V*) = O(n v max ). V* = optimal value = maximum v such that OPT(n, v) W. Not polynomial in input size! V* n v max Copyright 000, Kevin Wayne

3 Knapsack: FPTAS Intuition for approximation algorithm. Round all values up to lie in smaller range. Run dynamic programming algorithm on rounded instance. Return optimal items in rounded instance. Knapsack: FPTAS Knapsack FPTAS. Round up all values: v i v i Theorem. If S is solution found by our algorithm and S* is any other feasible solution then () v i v i i S i S* Item Value Weight 934, 5,956,34 3 7,80,03 5 4,7, ,343,99 7 W = original instance Item Value 6 Weight W = rounded instance Pf. Let S* be any feasible solution satisfying weight constraint. v i i S* v i i S* v i i S (v i ) i S v i n i S () v i i S always round up solve rounded instance optimally never round up by more than S n n = v max, v max is v i DP alg can take v max 9 Knapsack: FPTAS Knapsack FPTAS. Round up all values: v max = largest value in original instance = precision parameter = scaling factor = v max / n v i v i, v ˆ i v i Observation. Optimal solution to problems with or are equivalent. v v ˆ Chapter 3 Randomized Algorithms v v Intuition. close to v so optimal solution using is nearly optimal; small and integral so dynamic programming algorithm is fast. v ˆ Running time. O(n 3 / ). Dynamic program II running time is O(n v ˆ max ), where v ˆ max v max n Slides by Kevin Wayne. 005 Pearson-Addison Wesley. All rights reserved. 0 Copyright 000, Kevin Wayne 3

4 Randomization Contention Resolution in a Distributed System Algorithmic design patterns. Greedy. Divide-and-conquer. Dynamic programming. Network flow. Randomization. in practice, access to a pseudo-random number generator Randomization. Allow fair coin flip in unit time. Contention resolution. Given n processes P,, P n, each competing for access to a shared database. If two or more processes access the database simultaneously, all processes are locked out. Devise protocol to ensure all processes get through on a regular basis. Restriction. Processes can't communicate. Challenge. Need symmetry-breaking paradigm. Why randomize? Can lead to simplest, fastest, or only known algorithm for a particular problem. P Ex. Symmetry breaking protocols, graph algorithms, quicksort, hashing, load balancing, Monte Carlo integration, cryptography. P... P n 3 5 Contention Resolution: Randomized Protocol 3. Contention Resolution Protocol. Each process requests access to the database at time t with probability p = /n. Claim. Let S[i, t] = event that process i succeeds in accessing the database at time t. Then /(e n) Pr[S(i, t)] /(n). Pf. By independence, Pr[S(i, t)] = p (-p) n-. process i requests access none of remaining n- processes request access Setting p = /n, we have Pr[S(i, t)] = /n ( - /n) n-. value that maximizes Pr[S(i, t)] between /e and / Useful facts from calculus. As n increases from, the function: ( - /n) n- converges monotonically from /4 up to /e ( - /n) n- converges monotonically from / down to /e. 6 Copyright 000, Kevin Wayne 4

5 Contention Resolution: Randomized Protocol Claim. The probability that process i fails to access the database in en rounds is at most /e. After en(c ln n) rounds, the probability is at most n -c. 3. Global Minimum Cut Pf. Let F[i, t] = event that process i fails to access database in rounds through t. By independence and previous claim, we have Pr[F(i, t)] ( - /(en)) t. Choose t = e n: en en Pr[F(i, t)] en en e Choose t = e n c ln n: Pr[F(i, t)] c ln n e n c 7 Contention Resolution: Randomized Protocol Global Minimum Cut Claim. The probability that all processes succeed within e n ln n rounds is at least - /n. Pf. Let F[t] = event that at least one of the n processes fails to access database in any of the rounds through t. Pr F[t] n Pr F[i, t ] i Pr[ F[i, t]] n Choosing t = en c ln n yields Pr[F[t]] n n - = /n. n i union bound previous slide en t Global min cut. Given a connected, undirected graph G = (V, E) find a cut (A, B) of minimum cardinality. Applications. Partitioning items in a database, identify clusters of related documents, network reliability, network design, circuit design, TSP solvers. Network flow solution. Replace every edge (u, v) with two antiparallel edges (u, v) and (v, u). Pick some vertex s and compute min s-v cut separating s from each other vertex v V. Union bound. Given events E,, E n, Pr n i E i n Pr[E i ] i False intuition. Global min-cut is harder than min s-t cut. 8 0 Copyright 000, Kevin Wayne 5

6 Contraction Algorithm Contraction Algorithm Contraction algorithm. [Karger 995] Pick an edge e = (u, v) uniformly at random. Contract edge e. replace u and v by single new super-node w preserve edges, updating endpoints of u and v to w keep parallel edges, but delete self-loops Repeat until graph has just two nodes v and v. Return the cut (all nodes that were contracted to form v ). Claim. The contraction algorithm returns a min cut with prob /n. Pf. Consider a global min-cut (A*, B*) of G. Let F* be edges with one endpoint in A* and the other in B*. Let k = F* = size of min cut. Let G' be graph after j iterations. There are n' = n-j supernodes. Suppose no edge in F* has been contracted. The min-cut in G' is still k. Since value of min-cut is k, E' ½kn'. Thus, algorithm contracts an edge in F* with probability /n'. Let E j = event that an edge in F* is not contracted in iteration j. a b c a b u d v contract u-v w e f f c 3 Pr[E E E n ] Pr[E ] Pr[E E ] Pr[E n E E E n3 ] n n 4 3 n n n(n) n n3 n 4 3 Contraction Algorithm Contraction Algorithm Claim. The contraction algorithm returns a min cut with prob /n. Pf. Consider a global min-cut (A*, B*) of G. Let F* be edges with one endpoint in A* and the other in B*. Let k = F* = size of min cut. In first step, algorithm contracts an edge in F* probability k / E. Every node has degree k since otherwise (A*, B*) would not be min-cut. E ½kn. Thus, algorithm contracts an edge in F* with probability /n. Amplification. To amplify the probability of success, run the contraction algorithm many times. Claim. If we repeat the contraction algorithm n ln n times with independent random choices, the probability of failing to find the global min-cut is at most /n. Pf. By independence, the probability of failure is at most A* B* n n lnn n n lnn e lnn n ( - /x) x /e F* 4 Copyright 000, Kevin Wayne 6

7 Global Min Cut: Context Expectation Remark. Overall running time is slow since we perform (n log n) iterations and each takes (m) time. Improvement. [Karger-Stein 996] O(n log 3 n). Early iterations are less risky than later ones: probability of contracting an edge in min cut hits 50% when n / nodes remain. Run contraction algorithm until n / nodes remain. Run contraction algorithm twice on resulting graph, and return best of two cuts. Extensions. Naturally generalizes to handle positive weights. Expectation. Given a discrete random variables X, its expectation E[X] is defined by: E[X] j Pr[X j] j0 Waiting for a first success. Coin is heads with probability p and tails with probability -p. How many independent flips X until first heads? E[X] j Pr[X j] j ( p) j p p j ( p) j p p p p p p j0 j0 j- tails head j0 Best known. [Karger 000] O(m log 3 n). faster than best known max flow algorithm or deterministic global min cut algorithm 5 7 Expectation: Two Properties 3.3 Linearity of Expectation Useful property. If X is a 0/ random variable, E[X] = Pr[X = ]. Pf. E[X] j Pr[X j] j Pr[X j] Pr[X ] j0 j0 not necessarily independent Linearity of expectation. Given two random variables X and Y defined over the same probability space, E[X + Y] = E[X] + E[Y]. Decouples a complex calculation into simpler pieces. 8 Copyright 000, Kevin Wayne 7

8 Guessing Cards Coupon Collector Game. Shuffle a deck of n cards; turn them over one at a time; try to guess each card. Memoryless guessing. No psychic abilities; can't even remember what's been turned over already. Guess a card from full deck uniformly at random. Claim. The expected number of correct guesses is. Pf. (surprisingly effortless using linearity of expectation) Let X i = if i th prediction is correct and 0 otherwise. Let X = number of correct guesses = X + + X n. E[X i ] = Pr[X i = ] = /n. E[X] = E[X ] + + E[X n ] = /n + + /n =. linearity of expectation Coupon collector. Each box of cereal contains a coupon. There are n different types of coupons. Assuming all boxes are equally likely to contain each coupon, how many boxes before you have coupon of each type? Claim. The expected number of steps is (n log n). Pf. Phase j = time between j and j+ distinct coupons. Let X j = number of steps you spend in phase j. Let X = number of steps in total = X 0 + X + + X n-. n n n n E[X] E[X j ] n n j i j0 j0 i prob of success = (n-j)/n expected waiting time = n/(n-j) nh(n) 9 3 Guessing Cards Game. Shuffle a deck of n cards; turn them over one at a time; try to guess each card. 3.4 MAX 3-SAT Guessing with memory. Guess a card uniformly at random from cards not yet seen. Claim. The expected number of correct guesses is (log n). Pf. Let X i = if i th prediction is correct and 0 otherwise. Let X = number of correct guesses = X + + X n. E[X i ] = Pr[X i = ] = / (n - i - ). E[X] = E[X ] + + E[X n ] = /n + + / + / = H(n). linearity of expectation ln(n+) < H(n) < + ln n 30 Copyright 000, Kevin Wayne 8

9 Maximum 3-Satisfiability The Probabilistic Method exactly 3 distinct literals per clause MAX-3SAT. Given 3-SAT formula, find a truth assignment that satisfies as many clauses as possible. C x x 3 x 4 Corollary. For any instance of 3-SAT, there exists a truth assignment that satisfies at least a 7/8 fraction of all clauses. Pf. Random variable is at least its expectation some of the time. C x x 3 x 4 C 3 x x x 4 C 4 x x x 3 Remark. NP-hard search problem. C 5 x x x 4 Probabilistic method. We showed the existence of a nonobvious property of 3-SAT by showing that a random construction produces it with positive probability! Simple idea. Flip a coin, and set each variable true with probability ½, independently for each variable Maximum 3-Satisfiability: Analysis Maximum 3-Satisfiability: Analysis Claim. Given a 3-SAT formula with k clauses, the expected number of clauses satisfied by a random assignment is 7k/8. Pf. Consider random variable Let Z = weight of clauses satisfied by assignment Z j. linearity of expectation E[Z] E[Z j ] Z j if clause C j is satisfied 0 otherwise. k j k Pr[clause C j is satisfied] j 8 7 k Q. Can we turn this idea into a 7/8-approximation algorithm? In general, a random variable can almost always be below its mean. Lemma. The probability that a random assignment satisfies 7k/8 clauses is at least /(8k). Pf. Let p j be probability that exactly j clauses are satisfied; let p be probability that 7k/8 clauses are satisfied. 7 8 jp j j 0 jp j jp j j 7k /8 j 7k /8 ( 7k 8 ) p 8 j k p j j 7k /8 j 7k /8 ( 8 7 k ) kp Rearranging terms yields p / (8k). Copyright 000, Kevin Wayne 9

10 Maximum 3-Satisfiability: Analysis Monte Carlo vs. Las Vegas Algorithms Johnson's algorithm. Repeatedly generate random truth assignments until one of them satisfies 7k/8 clauses. Theorem. Johnson's algorithm is a 7/8-approximation algorithm. Monte Carlo algorithm. Guaranteed to run in poly-time, likely to find correct answer. Ex: Contraction algorithm for global min cut. Pf. By previous lemma, each iteration succeeds with probability at least /(8k). By the waiting-time bound, the expected number of trials to find the satisfying assignment is at most 8k. Las Vegas algorithm. Guaranteed to find correct answer, likely to run in poly-time. Ex: Randomized quicksort, Johnson's MAX-3SAT algorithm. stop algorithm after a certain point Remark. Can always convert a Las Vegas algorithm into Monte Carlo, but no known method to convert the other way Maximum Satisfiability RP and ZPP Extensions. Allow one, two, or more literals per clause. Find max weighted set of satisfied clauses. Theorem. [Asano-Williamson 000] There exists a approximation algorithm for MAX-SAT. Theorem. [Karloff-Zwick 997, Zwick+computer 00] There exists a 7/8-approximation algorithm for version of MAX-3SAT where each clause has at most 3 literals. Theorem. [Håstad 997] Unless P = NP, no -approximation algorithm for MAX-3SAT (and hence MAX-SAT) for any > 7/8. RP. [Monte Carlo] Decision problems solvable with one-sided error in poly-time. One-sided error. If the correct answer is no, always return no. If the correct answer is yes, return yes with probability ½. ZPP. [Las Vegas] Decision problems solvable in expected polytime. running time can be unbounded, but on average it is fast Theorem. P ZPP RP NP. Can decrease probability of false negative to -00 by 00 independent repetitions very unlikely to improve over simple randomized algorithm for MAX-3SAT Fundamental open questions. To what extent does randomization help? Does P = ZPP? Does ZPP = RP? Does RP = NP? Copyright 000, Kevin Wayne 0

11 Hashing 3.6 Universal Hashing Hash function. h : U { 0,,, n- }. Hashing. Create an array H of size n. When processing element u, access array element H[h(u)]. Collision. When h(u) = h(v) but u v. A collision is expected after (n) random insertions. This phenomenon is known as the "birthday paradox." Separate chaining: H[i] stores linked list of elements u with h(u) = i. H[] jocularly seriously H[] null H[3] suburban untravelled considerating H[n] browsing 43 Dictionary Data Type Ad Hoc Hash Function Dictionary. Given a universe U of possible elements, maintain a subset S U so that inserting, deleting, and searching in S is efficient. Dictionary interface. Create(): Initialize a dictionary with S =. Insert(u): Add element u U to S. Delete(u): Delete u from S, if u is currently in S. Lookup(u): Determine whether u is in S. Challenge. Universe U can be extremely large so defining an array of size U is infeasible. Applications. File systems, databases, Google, compilers, checksums PP networks, associative arrays, cryptography, web caching, etc. Ad hoc hash function. int h(string s, int n) { int hash = 0; for (int i = 0; i < s.length(); i++) hash = (3 * hash) + s[i]; return hash % n; } hash function ala Java string library Deterministic hashing. If U n, then for any fixed hash function h, there is a subset S U of n elements that all hash to same slot. Thus, (n) time per search in worst-case. Q. But isn't ad hoc hash function good enough in practice? 4 44 Copyright 000, Kevin Wayne

12 Algorithmic Complexity Attacks Universal Hashing When can't we live with ad hoc hash function? Obvious situations: aircraft control, nuclear reactors. Surprising situations: denial-of-service attacks. malicious adversary learns your ad hoc hash function (e.g., by reading Java API) and causes a big pile-up in a single slot that grinds performance to a halt Universal class of hash functions. [Carter-Wegman 980s] For any pair of elements u, v U, Pr h H h(u) h(v) /n Can select random h efficiently. Can compute h(u) efficiently. Ex. U = { a, b, c, d, e, f }, n =. chosen uniformly at random Real world exploits. [Crosby-Wallach 003] Bro server: send carefully chosen packets to DOS the server, using less bandwidth than a dial-up modem Perl 5.8.0: insert carefully chosen strings into associative array. Linux.4.0 kernel: save files with carefully chosen names. h (x) h (x) h (x) h (x) h 3 (x) h 4 (x) a b c d e f a b c d e f H = {h, h } Pr h H [h(a) = h(b)] = / Pr h H [h(a) = h(c)] = Pr h H [h(a) = h(d)] = 0... H = {h, h, h 3, h 4 } Pr h H [h(a) = h(b)] = / Pr h H [h(a) = h(c)] = / Pr h H [h(a) = h(d)] = / Pr h H [h(a) = h(e)] = / Pr h H [h(a) = h(f)] = 0... not universal universal Hashing Performance Universal Hashing Idealistic hash function. Maps m elements uniformly at random to n hash slots. Running time depends on length of chains. Average length of chain = = m / n. Choose n m on average O() per insert, lookup, or delete. Challenge. Achieve idealized randomized guarantees, but with a hash function where you can easily find items where you put them. Approach. Use randomization in the choice of h. adversary knows the randomized algorithm you're using, but doesn't know random choices that the algorithm makes Universal hashing property. Let H be a universal class of hash functions; let h H be chosen uniformly at random from H; and let u U. For any subset S U of size at most n, the expected number of items in S that collide with u is at most. Pf. For any element s S, define indicator random variable X s = if h(s) = h(u) and 0 otherwise. Let X be a random variable counting the total number of collisions with u. E hh [X] E[ ss X s ] ss E[X s ] ss Pr[X s ] ss n S n linearity of expectation X s is a 0- random variable universal (assumes u S) Copyright 000, Kevin Wayne

13 Designing a Universal Family of Hash Functions Number Theory Facts Theorem. [Chebyshev 850] There exists a prime between n and n. Modulus. Choose a prime number p n. Integer encoding. Identify each element u U with a base-p integer of r digits: x = (x, x,, x r ). Hash function. Let A = set of all r-digit, base-p integers. For each a = (a, a,, a r ) where 0 a i < p, define h a (x) Hash function family. H = { h a : a A }. r a i x i mod p i no need for randomness here Fact. Let p be prime, and let z 0 mod p. Then z = m mod p has at most one solution 0 < p. Pf. Suppose and are two different solutions. Then ( - )z = 0 mod p; hence ( - )z is divisible by p. Since z 0 mod p, we know that z is not divisible by p; it follows that ( - ) is divisible by p. This implies =. Bonus fact. Can replace "at most one" with "exactly one" in above fact. Pf idea. Euclid's algorithm Designing a Universal Class of Hash Functions Theorem. H = { h a : a A } is a universal class of hash functions. Pf. Let x = (x, x,, x r ) and y = (y, y,, y r ) be two distinct elements of U. We need to show that Pr[h a (x) = h a (y)] /n. Since x y, there exists an integer j such that x j y j. We have h a (x) = h a (y) iff a j ( y j x j ) a i (x i y i ) mod p i j z m Can assume a was chosen uniformly at random by first selecting all coordinates a i where i j, then selecting a j at random. Thus, we can assume a i is fixed for all coordinates i j. Since p is prime, a j z = m mod p has at most one solution among p possibilities. see lemma on next slide Thus Pr[h a (x) = h a (y)] = /p /n. 3.9 Chernoff Bounds 50 Copyright 000, Kevin Wayne 3

14 Chernoff Bounds (above mean) Chernoff Bounds (below mean) Theorem. Suppose X,, X n are independent 0- random variables. Let X = X + + X n. Then for any E[X] and for any > 0, we have Pr[ X ( ) e ] ( ) sum of independent 0- random variables is tightly centered on the mean Pf. We apply a number of simple transformations. For any t > 0, Pr[X ()] Pre tx e t() e t() E[e tx ] Theorem. Suppose X,, X n are independent 0- random variables. Let X = X + + X n. Then for any E[X] and for any 0 < <, we have Pf idea. Similar. Pr[ X ( ) ] e / Remark. Not quite symmetric since only makes sense to consider <. f(x) = e tx is monotone in x Markov's inequality: Pr[X > a] E[X] / a Now E[e tx ] E[e t i X i ] E[e tx i ] i definition of X independence Chernoff Bounds (above mean) Pf. (cont) Let p i = Pr[X i = ]. Then, 3.0 Load Balancing E[ e t Xi ] t i p e ( p ) e i 0 p ( e ) i t e p ( t i e ) for any 0, + e Combining everything: Pr[X ()] e t() E[e tx i i ] e t() e p i (e t ) i e t() e (et ) previous slide inequality above i p i = E[X] Finally, choose t = ln( + ). 54 Copyright 000, Kevin Wayne 4

15 Load Balancing Load Balancing: Many Jobs Load balancing. System in which m jobs arrive in a stream and need to be processed immediately on n identical processors. Find an assignment that balances the workload across processors. Centralized controller. Assign jobs in round-robin manner. Each processor receives at most m/n jobs. Decentralized controller. Assign jobs to processors uniformly at random. How likely is it that some processor is assigned "too many" jobs? Theorem. Suppose the number of jobs m = 6n ln n. Then on average, each of the n processors handles = 6 ln n jobs. With high probability every processor will have between half and twice the average load. Pf. Let X i, Y ij be as before. Applying Chernoff bounds with = yields 6n ln n ln n e Pr[ X i ] 4 e n Pr[ X i ] e (6n lnn) n Union bound every processor has load between half and twice the average with probability - /n Load Balancing Analysis. Let X i = number of jobs assigned to processor i. Let Y ij = if job j assigned to processor i, and 0 otherwise. We have E[Y ij ] = /n Thus, X i = j Y i j,and = E[X i ] =. c e Applying Chernoff bounds with = c - yields Pr[ X i c] c c Extra Slides Let (n) be number x such that x x = n, and choose c = e (n). c c e Pr[ Xi c] c c e c ( n) e ( n) ( n) ( n) n Union bound with probability - /n no processor receives more than e (n) = (logn / log log n) jobs. Fact: this bound is asymptotically tight: with high probability, some processor receives (logn / log log n) 58 Copyright 000, Kevin Wayne 5

16 Quicksort 3.5 Randomized Divide-and-Conquer Running time. [Best case.] Select the median element as the splitter: quicksort makes (n log n) comparisons. [Worst case.] Select the smallest element as the splitter: quicksort makes (n ) comparisons. Randomize. Protect against worst case by choosing splitter at random. Intuition. If we always select an element that is bigger than 5% of the elements and smaller than 5% of the elements, then quicksort makes (n log n) comparisons. Notation. Label elements so that x < x < < x n. 63 Quicksort Quicksort: BST Representation of Splitters Sorting. Given a set of n distinct elements S, rearrange them in ascending order. RandomizedQuicksort(S) { if S = 0 return BST representation. Draw recursive BST of splitters. x 7 x 6 x x 3 x x 8 x 7 x x 5 x 3 x 7 x 0 x 6 x 4 x 9 x 4 x 5 } choose a splitter a i S uniformly at random foreach (a S) { if (a < a i ) put a in S - else if (a > a i ) put a in S + } RandomizedQuicksort(S - ) output a i RandomizedQuicksort(S + ) x 5 S - S + x 3 x 9 first splitter, chosen uniformly at random x 0 x x 3 x 6 x x 4 x 7 x x 5 x 7 Remark. Can implement in-place. O(log n) extra space x x 6 x 8 x Copyright 000, Kevin Wayne 6

17 Quicksort: BST Representation of Splitters Observation. Element only compared with its ancestors and descendants. x and x 7 are compared if their lca = x or x 7. x and x 7 are not compared if their lca = x 3 or x 4 or x 5 or x 6. Claim. Pr[x i and x j are compared] = / j - i +. x 0 x 5 x 3 x 3 x 9 x x 6 x x 4 x 7 x x 5 x 7 x x 6 x 8 x 4 65 Quicksort: Expected Number of Comparisons Theorem. Expected # of comparisons is O(n log n). Pf. j i n i n n n n j j x dx n ln n i j n i j j x probability that i and j are compared Theorem. [Knuth 973] Stddev of number of comparisons is ~ 0.65N. Ex. If n = million, the probability that randomized quicksort takes less than 4n ln n comparisons is at least 99.94%. Chebyshev's inequality. Pr[ X - k] / k. 66 Copyright 000, Kevin Wayne 7