Chapter 6. Self-Adjusting Data Structures

Chapter 6 Self-Adjusting Data Structures Chapter 5 describes a data structure that is able to achieve an epected quer time that is proportional to the entrop of the quer distribution. The most interesting aspect of this data structure is that it does need to know the quer distribution in advance. Rather, the data structure adjusts itself using information gathered from previous queries. In this chapter we stud several other data structures that adjust themselves in the same wa. All of these data structures are ecient onl in an amortized sense. That is, individual operations take ma take a large amount of time, but over a sequence of man operations, the amortized (average) time per operation is still small. The advantage of self-organizing data structures is that the can often perform \natural" sequences of accesses much faster than their worst-case counterparts. Eamples of such natural sequences including accessing each of the elements in order, using the data structure like a stack or queue and performing man consecutive accesses to the same element. In other sections of this book we have analzed the amortized cost of operations on data structures using the accounting method, in which we manipulate credits to bound the amortized cost of operations. A dierent method of amortized analsis is the potential method. For an data structure D, we dene a potential Φ(D). If we let D i denote our data structure after the ith operation and let C i denote the cost (running time) of the ith operation then the amortized cost of the ith operation is dened as A i = C i + Φ(D i ) = Φ(D i 1 ). Note that this is simpl a denition and, since Φ is arbitrar, A i has nothing to do with the real cost 32

CHAPTER 6. SELF-ADJUSTING DATA STRUCTURES 33 C i. However, the the cost of a sequence of m operations is C i = (C i + Φ(D i ) Φ(D i 1 )) (Φ(D i ) Φ(D i 1 )) = = A i (Φ(D i ) Φ(D i 1 )) A i + Φ(D 0 ) Φ(D m ). The quantit Φ(D 0 ) Φ(D m ) is referred to as the net drop in potential. Thus, if we can provide a bound D on A i, then the total cost of m operations is md plus the net drop in potential. The potential method is equivalent to the accounting method. Anthing that can be proven with the potential method can be proven with the accounting method. (Just think of Φ(D) as the number of credits the data structure stores, with a negative value indicating that the data structure owes credits) However, once a potential function is dened, mathematical manipulations that can lead to ver compact proofs are convenient. Of course, the rub is in nding the right potential function. 6.1 Spla Trees One of the rst-ever self-modifing data structures is the spla tree. We begin our discussion of spla trees with the denition and analsis of the spla operation. A spla tree is a binar search tree that is augmented with the spla operation. Given a binar search tree T rooted at r and a node in T, splaing node is a sequence of left and right rotations performed on the tree until is the root of T. This sequence of rotations is governed b the following rules: (Note: a node is a left (respectivel, right) child if it is the left (respectivel, right) child of its parent) In what follows, is the parent of and z is the parent of (should it eist). 1. Zig step: If is the root of the tree, then rotate so that it becomes the root. 2. Zig Zig step: The node is not the root. Both and are left (or right) children, then rst rotate, so that z and are children of, then rotate so that it becomes the parent of. 3. Zig Zag step: Node is not the root. If is a right child and is a left child or vice-versa, rotate twice. After one rotation it is the parent of and after the second rotation, both and z are children of. We are now read to dene the potential of a spla tree. Let T be a binar search tree on n nodes and let the nodes be numbered from 1 to n. Assume that each node i has a ed positive weight w(i). Dene the size s() of a node to be the sum of the weights of all the nodes in the subtree rooted at, including the weight of. The rank r() of a node is log(s()). B denition, the rank of a node is at least as large as the rank of an of its decendants. The potential of a given tree T is Φ(T) = n log(s(i)). All of the properties of a spla tree are derived from the following lemma:

CHAPTER 6. SELF-ADJUSTING DATA STRUCTURES 34 zig z zig-zig z z zig-zag z Figure 6.1: Spla tree. Lemma 3. (Access Lemma) The amortized time to spla a binar search tree T with root t at a node is at most 3(r(t) r()) + 1 = O(log(s(t))/ log(s())) Proof. If node is the root, then the lemma holds triviall. If node is not the root, then a number of rotations are performed according to the rules of splaing. We rst show that the lemma holds after each step (zig, zigzig or zigzag) is performed. Let r 0 (i), r 1 (i) and s 0 (i), s 1 (i) represent the rank and size of node i before and after the rst step, respectivel. Recall that the amortized cost of an operation a = t + Φ 1 (T) Φ 0 (T) where t is the actual cost of the operation, and Φ 0 and Φ 1 are the potentials before and after the step. 1. Zig step: In this case, is the root of the tree and a single rotation is performed to make the root of the tree. In the whole spla operation, this step is onl ever eecuted once. The actual cost of the rotation is 1. We show that the amortized cost is at most 3(r 1 () r 0 ()) + 1. amortized cost = 1 + r 1 () + r 1 () r 0 () r 0 () (6.1) 1 + r 1 () r 0 () (6.2) 1 + 3(r 1 () r 0 ()) (6.3) Equation (6.1) follows from the denition of amortized cost and the fact that the onl ranks that changed are the ranks at and. The ranks of all other nodes in the tree remain the same since the nodes in their subtrees remain the same. Therefore, the change in potential is determined b the change in the ranks of and. Equation (6.2) holds because prior to the rotation, node was the root of the tree which implies that r 0 () r 1 (). Equation (6.3) follows since after the rotation, node is the root the tree, which means that r 1 () r 0 () is positive.

CHAPTER 6. SELF-ADJUSTING DATA STRUCTURES 35 2. Zig Zig step: The node is not the root. Both and z are left (or right) children. In this case, two rotations are made. The rst rotation is at, so that z and are children of. The second rotation is at so that it becomes the parent of and grandparent of z. Since two rotations are performed, the actual cost of this step is 2. amortized cost = 2 + r 1 () + r 1 () + r 1 (z) r 0 () r 0 () r 0 (z) (6.4) = 2 + r 1 () + r 1 (z) r 0 () r 0 () (6.5) 2 + r 1 () + r 1 (z) 2r 0 ()) (6.6) After the two rotations, the onl ranks that change are the ranks of, and z. Therefore, the change in potential is dened b the change in their ranks. Equation (6.4) follows from the denition of amortized cost. Now, prior to the two rotations, is child of which is a child of z and after the two rotations, z is a child of which is a child of. This implies that r 1 () = r 0 (z). Equation (6.5) follows. Similarl, (6.6) follows since r 1 () r 1 () and r 0 () r 0 (). What remains to be shown is that 2 + r 1 () + r 1 (z) 2r 0 ()) 3(r 1 () r 0 ()). With a little manipulation, we see that this is equivalent to showing that r 1 (z)+r 0 () 2r 1 () 2. (6.7) r 1 (z) + r 0 () 2r 1 () = log(s 1 (z)/s 1 ()) + log(s 0 ()/s 1 ()) (6.8) = log((s 1 (z)/s 1 ()) (s 0 ()/s 1 ())) (6.9) log(1/4) (6.10) = 2 (6.11) Equation (6.8) simpl follows from the denition of ranks. Notice that s 1 () s 1 (z) + s 0 (). This implies that s 1 (z)/s 1 () + s 0 ()/s 1 () 1. If we let b = s 1 (z)/s 1 () and c = s 0 ()/s 1 (), we note that b + c 1 and the epression in (6.9) becomes log(bc). Elementar calculus shows that given b + c 1, the product bc is maimized when b = c = 1/2. This gives (6.10). 3. Zig Zag step: Node is not the root. If is a right child and is a left child or vice-versa, rotate twice. After one rotation it is the parent of and after the second rotation, both and z are children of. Again since two rotations are performed, the actual cost of the operation is 2. Let us look at what the amortized cost is. amortized cost = 2 + r 1 () + r 1 () + r 1 (z) r 0 () r 0 () r 0 (z) (6.12) = 2 + r 1 () + r 1 (z) r 0 () r 0 () (6.13) 2 + r 1 () + r 1 (z) 2r 0 ()) (6.14) (6.15) After the two rotations, the onl ranks that change are the ranks of, and z. Therefore, the change in potential is again dened b the change in their ranks. Equation (6.12) follows from the denition of amortized cost. Now, prior to the two rotations, z is the root of the subtree under consideration and after the two rotations, is the root. Therefore, we have r 1 () = r 0 (z). Equation (6.13) follows. Similarl, r 0 () r 0 () since is a child of prior to the rotations. What remains to be shown is that 2 + r 1 () + r 1 (z) 2r 0 ()) 3(r 1 () r 0 ()). This is equivalent showing that r 1 () + r 1 (z) + r 0 () 3r 1 () 2.

CHAPTER 6. SELF-ADJUSTING DATA STRUCTURES 36 r 1 () + r 1 (z) + r 0 () 3r 1 () = log((s 1 ()/s 1 ()) (s 1 (z)/s 1 ())) + r 0 () r 1 () (6.16) log(1/4) (6.17) = 2 (6.18) The argument is similar to the previous case. Equation (6.16) simpl follows from the denition of ranks and some manipulation of logs. The inequalit (6.17) follows because s 1 () s 1 () + s 1 (z) and r 0 () r 1 (). Now, suppose that a spla operation requires k > 1 steps. Onl the kth step will be a zig step and all the others will either be zigzig or zigzag steps. From the three cases shown above, the amortized cost of the whole spla operation is k 1 3r k () 3r k 1 () + 1 + (3r j () 3r j 1 ()) = 3r k () 3r 0 () + 1. j=1 If each node is given a weight of 1, the above lemma suggests that the amortized cost of performing a spla operation is O(log n) where n is the size of the tree. However, Lemma 3 is surprisingl much more powerful. Notice that the weight of a node was never reall specied. We onl stated that it should be a positive constant. The power comes from the fact that the lemma holds for an assignment of positive weights to the nodes. B carefull choosing the weights, we can prove man interesting properties of spla trees. Let us look at some eamples, starting with the most mundane. Theorem 14 (Balance Theorem). Given a sequence of m accesses into an n node spla tree T, the total access time is O(n log n + m log n). Proof. Assign a weight of 1 to each node. The maimum value that s() can have for an node 2 T is n since the sum of the weights is n. The minimum value of s() is 1. Therefore, the amortized access cost for an element is at most 3(log(n) log(1)) + 1 = 3 log(n) + 1. The maimum potential Φ ma (T) that the tree can have is log(s(i)) log(n) = n log n The minimum potential Φ min (T) that the tree can have is log(s(i)) log(1) = 0. Therefore, the maimum drop in potential over the whole sequence is Φ ma (T) Φ min (T) = n log n. Thus, the total cost of accessing the sequence is O(m log n + n log n), as required. This means that starting from an initial binar search tree, if we access m > n kes, the cost per access will be O(log n). This is quite interesting since we can start with a ver unbalanced tree, such as an n-node path, and still achieve good behaviour. Net we prove something much more interesting.

CHAPTER 6. SELF-ADJUSTING DATA STRUCTURES 37 Theorem 15 (Static Optimalit Meta-Theorem). Fi some static binar search tree T and let d() denote the depth of the node containing in T. For an access sequence a 1,..., a m consisting of elements in T, the cost of accessing a 1,..., a m using a spla tree is at most O(n log n + m + m d(a i). Proof. If d() is the depth of node in T, then we assign a weight of w() = 1/n + 1/3 d() to the node containing in the spla tree. Now, note that the size of the root (and hence an node) is at most 1/n + 1/3 d() = 1 + 1/3 d() 1 + 2 i /3 i = 4, 2T i=0 2T since T has at most 2 i nodes with depth j and these each have weight 1/3 j. Now, appling the Access Lemma, we nd that the amortized cost of accessing node is at most 3(log 4 log(s())) + 1 = 7 3 log(1/n + 1/3 d() ) 7 3 log(1/3 d() ) = 7 + 3 log(3 d() ) = 7 + (3 log 3)d() All that remains is to bound the net drop in potential, Φ ma Φ min. We have alread determined that s() 4 for an node, so r() 2 and Φ ma 2n. On the other hand, s() w() 1/n for ever node, so r() log n and Φ min n log n. Putting everthing together, we get that the total cost of accessing a 1,..., a m using a spla tree is C(a 1,..., a m ) Φ ma Φ min + = 2n ( n log n) + (7 + (3 log 3)d(a i )) (7 + (3 log 3)d(a i )) = n log n + 2n + 7m + (3 log 3) = O n log n + m +! d(a i ) d(a i ). The Static Optimalit Meta-Theorem allows us to compare the performance of a spla tree with that of an ed binar search tree. For eample, letting T be a perfectl balanced binar search tree on n items so that d() log n for ever and appling the Static Optimalit Theorem gives the Balance Theorem. As another eample, in Chapter 5, we saw that when searches are drawn independentl from some distribution, then there eists a binar search tree whose epected search time is proportional to the entrop of that distribution. This implies the following corollar of the Static Optimalit Meta- Theorem:

CHAPTER 6. SELF-ADJUSTING DATA STRUCTURES 38 Corollar 1. (Static Optimalit Theorem) If a 1,..., a m are drawn from a set of size n, each independentl from the a probabilit distribution with entrop H, then the epected time to access a 1,..., a m using a spla tree is O(n log n + m + mh). As another eample, we obtain the static nger propert. As an eercise, show how, given an integer f 2 {1,..., n} to design a static tree that stores 1,..., n and such that the time to search for is O(1 + log f ). (Hint: Tr the special case f = 1 rst.) The eistence of this tree implies the following result: Corollar 2. (Static Finger Theorem) Let a 1,..., a m be a sequence of values in {1,..., n}. Then, for an f 2 {1,..., n}, the time to access a 1,..., a m using a spla tree is O(n log n+m+ m log a i f ). Of course, Corollar 2 is not reall restricted to trees that store {1,..., n}; these are just standins for the ranks of elements in an totall ordered set. We nish with one additional propert of spla trees that is not covered b the Static Optimalit Theorem: Theorem 16. (Working-Set Theorem) For an access sequence a 1,..., a m the cost of accessing a 1,..., a m using a spla tree is at most O(n log n + m + m log w i(a i )), where { min{j 1 : a i j = } if 2 {a 1,..., a i 1 } w i () = n otherwise. Proof. Assign the weights 1, 1/4, 1/9,..., 1/n 2 to the values in the spla tree in the order of their rst appearance in the sequence a 1,..., a m, so that w(a 1 ) = 1 and, if a 2 6= a 1, then w(a 2 ) = 1/4, and so on. Immediatel after an accessing and splaing an element a i, whose weight is 1/k 2, we will reweight some elements as follows: a i will be assigned weight 1, the item with weight 1 will be assigned weight 1/4, the item with weight 1/4 will be assigned weight 1/9 and, in general, an item with weight 1/j 2, j < k will be assigned weight 1/(j + 1) 2. Notice that, with this scheme, the sum of all the weights, and hence the size of the root is 1/i 2 1/i 2 = π 2 /6. (The second, innite, sum is the subject of the Basel Problem, rst solved b Euler in 1735; the solution can be found in most calculus tetbooks.) This initial assignment of weights along with the reassignment of weights ensures that the weight of a i at the time it is accessed is 1/(w i (a i )) 2. The Access Lemma then implies that the amortized cost to access a i is at most 3(log(π 2 /6) log(1/w i (a i ) 2 )) + 1 = 3(log(π 2 /6) + 2 log w i (a i )) + 1 This is all great, ecept that we also have to account for an increase in potential caused b the reweighting. The reweighting increases the weight of a i from 1/(w i (a i )) 2 to 1. For values other than a i, the weights onl decrease or remain unchanged. Since, b the time we do reweighting a i is at the root, the reweighting does not increase the rank of a i. On the other hand, it can onl decrease the ranks of other nodes, so overall it can not result in an increase in potential. To nish, we need onl bound Φ ma and Φ min. We alread have an upper bound of log(π 2 /6) on the rank of the root, which gives an upper bound of Φ ma n log(π 2 /6). For the lower-bound, note

CHAPTER 6. SELF-ADJUSTING DATA STRUCTURES 39 that ever node has weight at least 1/n 2, so it's rank is at least 2 log n and Φ min 2n log n. To summarize, the total cost of accessing a 1,..., a m is C(a 1,..., a m ) Φ ma Φ min + 3 log π 2 /6 + 2 log w i (a i ) = (π 2 /6)n + 2n log n + 3 log(π 2 /6)m + = O n log n + m +! log w i (a i ). 6 log w i (a i ) 6.2 Pairing Heaps John Iacono apparentl has a mapping between spla trees and pairing heaps that makes all the analses carr over. 6.3 Queaps Net, we consider an implementation of priorit queues. These are data structures that support the insertion of real-valued priorities. At an time, the data structure can return a pointer to a node storing the minimum priorit, and given a pointer to a node, that node (and it's corresponding priorit) can be deleted. For the data structure we describe, the cost to insert an element is O(1) and the cost to etract the element is O(log(q()), where q() denotes the number of elements that have been in the data structure longer than. Thus, if the priorit queue is used as a normal queue (the rst element inserted is the rst element deleted) then all operations take constant time. We sa that a data structure that has this propert is queueish. The data structure consists of a balanced tree T and a list L. Refer to Figure 6.3. The tree T is 2-4 tree, that is, all leaves of T are on the same level and each internal node of T has between 2 and 4 children. From this denition, it follows immediatel that the ith level of T contains at least 2 i nodes, so the height of T is logarithmic in the number of leaves. Another propert of 2-4 trees that is not immediatel obvious, but not dicult to prove, is that it is possible to insert and delete a leaf in O(1) amortized time. These insertions and deletions are done b splitting and merging nodes (see Figure 6.2), but an sequence of n insertions and deletions results onl in O(n) of these split and merge operations. It is important to note that T is not being used as a search tree. Rather, the leaves of T will store priorities so that the left to right order of the leaves is the same as the order in which the corresponding priorities were inserted. The nodes of T also contain some additional information. The left wall of T is the path from the root to the leftmost leaf. Each internal node v of T that is not part the left wall stores a pointer