Content. 1 Static dictionaries. 2 Integer data structures. 3 Graph data structures. 4 Dynamization and persistence. 5 Cache-oblivious structures

Transcription

1 Data strutures II NTIN067 Jira Fin fin/ Katedra teoreticé informatiy a matematicé logiy Matematico-fyziální faulta Univerzita Karlova v Praze Summer semester 08/9 Last change March, 09 License: Creative Commons BY-NC-SA 4.0 Jira Fin Data Structures II Jira Fin Data Structures II Jira Fin: Data Structures II General information fin@timl.mff.cuni.cz Homepage fin/ Consultations Individual schedule Jira Fin Data Structures II 3 Jira Fin Data Structures II 4 Computational model Word-RAM Static dictionaries Description A word is a w-bit integer A memory an array of words indexed by words The size of memory is w, so we assume that w = Ω(log n Operations of words in constant time: Arithmetical operations are +,,, /, mod Bit-wise operations &,,ˆ, >>, << Comparisons =, <,, >, Other operations in constant time: (unconditional jumps, assignments, memory accesses, etc. Inputs and outputs are stored in memory Notations Goal Universe U of all elements (words Store S U of size n in a data structure Using hashing, we store S in a table M = [m] = {0,..., m } of size m Create a data structure determining whether a given element of U belongs to S. Methods Build Member Search tree n log n log n optimal in the comparison model Cucoo n (exp. log n-independent FKS n (exp. -independent n log n deterministic Jira Fin Data Structures II 5 Jira Fin Data Structures II 6 Independent repeated trials Universal hashing systems Marov inequality If X is a non-negative random variable and c >, then P[X < ce[x]] > c c. Expected number of trial using probability Let V be an event that occurs in a trial with probability p. The expected number of trials to first occurrence of V in a sequence of independent trials is p. Expected number of trial using mean If X is a non-negative random variable and c >. The expected number of trials to first occurrence of X ce[x] in a sequence of independent trials is at most Example c c. If the expected number of collisions of a randomly chosen hashing function h is, then the expected number of independent trials to the first occurrence of a hashing function h with at most collisions is. c-universal hashing system A hashing system H of functions h : U M is c-universal for c > if a uniformly chosen h from H satifies P[h(x = h(y] c for every x, y U and x y. m -independent hashing system A hashing system H of functions h : U M is -independent for N if a uniformly chosen h from H satifies P[h(x i = z i for all i =,..., ] = O ( m for all pairwise different x,..., x U and all z,..., z M. Example: System Multiply-mod-prime Let p be a prime greater than U h a,b(x = (ax + b mod p mod m H = {h a,b; a, b [p], a 0} System H is -universal and -independent but it is not 3-independent Jira Fin Data Structures II 7 Jira Fin Data Structures II 8

2 Hagerup, Miltersen, Pagh, 00 [] Goal A static dictionary for n w-bit eys with constant looup time and a space consumption of O(n words can be constructed in O(n log n time on w-word-ram. The algorithm is wealy non-uniform, i.e. requires certain precomputed constants dependent on w. Overview Create a function f : [ w ] [ 4w ] which is an error-correcting code of relative minimum distance δ > 0. Create a function f : [ 4w ] [O ( n ] which is an injection on f (S Goal Find a function h : U [ r ] with U = [O ( n ] and r = max { w, 3 + log n} s.t. h is perfect on S U of size n and h can be computed in constant time and space consumption O(n for finding and storing h and h can be found in O(n log n worst case time. First, expected O(n time, then derandomize to O(n log n worst case time. 3 Create a function f 3 : [O ( n ] [O ( n ] which is an injection on f (f (S 4 Create a function f 4 : [O ( n ] [O(n] which is an injection on f 3(f (f (S f 4 f 3 f f can be computed in constant time f, f 3, f 4 can be found in time O(n log n f can be precomputed in time O(w x U is a point (α(x, β(x in a (O(n O(n-table For x U, let α(x denote the first r bits of x and β(x denotes the remaining bits. Then x (α(x, β(x is an injection (e.i. perfect on U. Jira Fin Data Structures II 9 Jira Fin Data Structures II 0 The number of remaining bits is at most r. Since r w. Definition For q 0 and functions f, g : U [ r ], the pair (f, g is q-good if f has at most q collisions and x (f (x, g(x is perfect on S. The number of collisions is the number of pairs {x, y} S such that f (x = f (y. Suppose that (f, g is q-good and r 3 + log n. Then, for every { v [ r ] there exists a v [ r ] such that (x g(x a f (x, f is q -good where q 0 if q n = n otherwise. All values a v can be computed in expected time O(n and space O(n worst case. Application: Randomized construction of a mapping [O ( n ] [O(n] 0 (α, β is ( n -good (x β(x a α(x, α =: (α, β is n-good (x β (x a α (x, α =: (α, β is 0-good, so α is perfect Jira Fin Data Structures II 0 Jira Fin Data Structures II Suppose that (f, g is q-good and r 3 + log n. Then, for every { v [ r ] there exists a v [ r ] such that (x g(x a f (x, f is q -good where q 0 if q n = n otherwise. All values a v can be computed in expected time O(n and space O(n worst case. Proof (q n Since x (f (x, g(x is perfect on S, g(x g(y. From g(x a f (x = g(x a f (y g(y a f (y it follows that h(x h(y. For every v [ r ] we randomly and independently choose a v from the uniform distribution on [ r ]. Then, h(x = h(y g(x a f (x = g(y a f (y a f (x = g(x g(y a f (y Since ([ r ], is an Abelian group, b b c is a bijection on [ r ] for every c [ r ] and so a f (y U[ r ], it follows that g(x g(y a f (y U[ r ]. Since a f (x and a f (y are independent, also a f (x and g(x g(y a f (y are independent. Hence, P[h(x = h(y] = r. 3 Use the linearity of expectation and substitute r. Let h(x = g(x a f (x If x, y S and x y and f (x = f (y, then g(x g(y and h(x h(y 3 If f (x f (y, then P[h(x = h(y] = where r av U[r ] independently for all v [ r ] 4 E[ {{x, y} S; h(x = h(y} ] ( n / r < n The expected number of trials to generate h with at most n collisions is O(. Jira Fin Data Structures II Jira Fin Data Structures II Suppose that (f, g is q-good and r 3 + log n. Then, for every { v [ r ] there exists a v [ r ] such that (x g(x a f (x, f is q -good where q 0 if q n = n otherwise. All values a v can be computed in expected time O(n and space O(n worst case. Proof (q n implies q = 0 Let S v = {x S; f (x = v} Order S v by non-increasing size, i.e. S v S v... S v r 3 For j =,..., r we find a vj such that h is perfect Note that we must find h without collisions. To be precise, we iteratively find a vj for j from to r such that it holds h(x h(y for every x S vj and y S <j where S <j = j i= Sv. i Linearity of expectation 3 Definition of S <j 4 S vi S vj 5 From this point, assume that S vj. 6 In order to verify that h has no collision, we use a counter m v = {y S <j; h(y = v}. For every j we can count the collisions and update m v in time O( S j. The expected time to find all a vj is j O( Sj = O(n. 7 For S vj = {x} we can find v with m v = 0 and set a vj = v g(x. 4 For a vj U[ r ] it holds E[ { (x, y S vj S <j; h(x = h(y } ] S vj S <j P[h(x = h(y] j i= Sv j Sv i /r 3 j i= S v i / r 4 j ( Svi i= / r 5 q/ r 5 The expected number of trials to generate a vj such that h has no collision is O(. 6 7 Jira Fin Data Structures II 3 Jira Fin Data Structures II 3

3 Derandomization Let L (a = { (x, y S vj S <j; (g(x a [] = (h(y [] } (a i denotes the i-th bit of a and (a M denotes the vector of all bits (a i for i M [r] for j to r do a vj 0 3 for 0 to r do 4 if L +(a vj > L +(a vj + then 5 a vj a vj + Where [r] and a [ ] Our goal is to iteratively and deterministically compute a vj for j from to r. The value of a v is computed by bits from the least significant to the most significant bit. L (a determines the number of collision between S vj and S <j if we consider only last bits. Proof (goodness of (h, f L (a + L (a = L (a for every a [ ] and [r] L (a vj L (av j L 0(av j = Sv j S <j The total number of collision is at most j Lr (av j j r S vj S <j i<j r S vj S vi ( r i Sv < n i 6 If q n, then the number of collision with S vj is L r (a vj i<j r S vj S vi ( i<j r Sv j r q Jira Fin Data Structures II 4 Jira Fin Data Structures II 4 Derandomization Let L (a = { (x, y S vj S <j; (g(x a [] = (h(y [] } (a i denotes the i-th bit of a and (a M denotes the vector of all bits (a i for i M [r] for j to r do a vj 0 3 for 0 to r do 4 if L +(a vj > L +(a vj + then 5 a vj a vj + Proof (Complexity In order to compute L (a, we build a binary tree (trie Every vertex a [ ] of the -th level has a counter c (a = { y S <j; (h(y [] = (a [] } L (a = x Sv j c (g(x a can be computed in O ( S vj time After the j-th step, counters can be updated in O ( S vj r time Total time is j Sv r = O(n log n j Static dictionaries: [O ( n ] [O ( n ] Approach Every x U = [O ( n ] can be regarded as constant-length string over an alphabet of size n Build n-way branching compressed trie of string S The number of leaves is S = n, so the total number of vertices is at most n Build static [O ( n ] [O(n] dictionary for pairs (vertex of the trie, letter which returns a child of the vertex One polynomial-size-universe looup is evaluated using a constant number of quadratic-size-universe looups Space complexity is O(n and this dictionary is constructed in O(n log n time Jira Fin Data Structures II 5 Jira Fin Data Structures II 6 Static dictionaries: Error-correcting code Definition The Hamming distance between x [ w ] and y [ w ] is the number of bits in which x and y differ. ψ : [ w ] [ 4w ] is an error correcting code of relative minimum distance δ > 0 if the Hamming distance between ψ(x and ψ(y is at least 4wδ for every distinct x, y [ w ]. For δ < it holds that 4wδ < and the identity is an error correcting code of 4w relative minimum distance δ. The number of y [ 4w ] within Hamming distance from a fixed x [ 4w ] is ( 4w i=0 i ( 4w ( 4w i=0 i ( 4w i ( 4w ( + 4w 4w ( 4w e ( 4ew using the binomial theorem. 3 By setting = 4wδ we obtain w 4w ( 4ew w ( 4ew 4wδ 4wδ = ( ( e δ 4δ w Let H be a -universal hashing system of function [ w ] [ 4w ]. For every δ with /4w < δ /, the probability that h U(H is an error correcting code of relative minimum distance δ > 0 is at least ( ( e δ 4δ /4 w. Proof For x [ 4w ] the number of y within Hamming distance is at most ( 4ew. For x y, P(Hamming distance between x and y 4w ( 4ew The probability that this happens for any of the ( w < w pairs is at most ( ( e δ 4δ /4 w 3 Jira Fin Data Structures II 7 Jira Fin Data Structures II 7 Static dictionaries: [ w ] [O ( n ] Let ψ : [ w ] [ 4w ] be an error correcting code of relative minimum distance δ > 0 and S U = [ w ] of size n. There exists a set D [4w] with D log n/ log δ such that for every pair x, y of distinct elements of S it holds (ψ(x D (ψ(y D. Proof Note that for every D [4w] the set S is split into D disjoint clusters C(D, v for v [ D ]. For i = 0 it holds that D 0 = and B(D 0 = ( n < n 3 A bit d [4w] \ D i with I(d ( δ B(D i can be found in O(wn time as follows. We a list of all clusters C(D i, v of size at least two. Every cluster has a list of all elements. So, I(d for one d [4w] \ D i can be determined in O(n time and we can process all d in O(wn time. Then, all lists can be updated in O(n time. Using word-level parallelism, the time complexity can be improved to O(n.. For D [4w] and v [ D ] let C(D, v = {x S; (ψ(x D = v} The set of colliding pairs of D is B(D = v [ D ] ( C(D,v We construct D 0 D... D such that D i = i and B(D i < ( δ i n / Let I(d = {{x, y} B(D i; (ψ(x d = (ψ(y d} be the colliding pairs indistinguishable by d [4w] \ D i Let I = d [4w]\D i I(d Every pair {x, y} B(D i contributes to I by at most 4w i 4wδ < 4w( δ, so I 4w( δ B(D i By averaging, there exists d [4w] \ D i such that I(d I ( δ B(D 4w i i 3 Let D i+ = D i {d}. Hence, B(D i+ = I(d ( δ B(D i By setting = log n/ log we obtain B(D δ <. Jira Fin Data Structures II 8 Jira Fin Data Structures II 8

4 Word- Theorem (Linear compression and speed up An algorithm running on a O(w-word-RAM with time complexity T (n and space complexity S(n can be simulated on w-word-ram with time complexity O(T (n and space complexity O(S(n assuming S(n = o( w. Proof Every word of O(w-word-RAM is replaced by O( words on w-word-ram, so the space complexity is O(T (n. Every instruction on O(w-word-RAM is replaced by O( instruction on w-word-ram. 3 Jira Fin Data Structures II 9 Jira Fin Data Structures II 0 This theorem is similar to the linear speed theorem for Turing machines. The assumption S(n = o( w is needed to ensure a sufficient amount of memory for linear data decompression. 3 Bit-wise operations and comparisons are trivial. In order to simulate arithmetical operations, every O(w bits word is split into w bits words to handle integer overflows and carry. RAM with uninitialized memory Theorem Every program with time complexity T (n and space complexity S(n which assumes initialized memory by zeros can be simulated by a program with time complexity O(T (n and space complexity S(n which do not assume initialized memory. Proof Simulation uses the following variables: M: An array of the original memory N: The number of already initialized cells by the original program I: An array of indices of all initialized cells X: A reverse table of I, i.e. if j is initialized, then I[X[j]] = j A read cell j is initialized if and only if X[j] < N and I[X[j]] = j. Jira Fin Data Structures II 0 Jira Fin Data Structures II Reading initialized memory must be permitted but an arbitrary value can be read. Integer data structures Goal Create a dynamic data structure storing a set S of intergers from a universe U with is can find the predecessor and the successor of a element x U. Predecessor: PRED(x U = max {y S; y < x} Successor SUCC(x U = min {y S; y > x} An invalid element is returned if no such element exists. Our nowledge O(log n query and update using search trees Jira Fin Data Structures II Jira Fin Data Structures II van Emde Boas tree [3] Notation For a t-bit integer x U let r(x be the left (upper t/ bits of x and l(x be the left (lower t/ bits of x. For a set of elements S let S i = {r(x S \ min S; l(x = i} for i l(u P = {S i; S i } A node of van Emde Boas tree contains Pointers to the minimum and maximum of S An array of veb trees for all sets S i A primary veb tree of elements P for finding non-empty bucets Space complexity Terrible: O(U, but we will improve it later. Requires initialized memory van Emde Boas tree: Operation FIND(x Algorithm Procedure FI N D(x, S if S = then 3 return None 4 if x = min(s then 5 return min(s 6 return FI N D(r(x, S l(x Analysis The height of the tree is O(log log U Time complexity is O(log log U Jira Fin Data Structures II 3 Jira Fin Data Structures II 4

5 van Emde Boas tree: Operation SUCC(x Algorithm Procedure SU C C(x, S if S = or x max(s then 3 return None 4 if x < min(s then 5 return min(s 6 i l(x 7 if S i and x < max(s i then 8 return SU C C(r(x, S i 9 i SU C C(i, P 0 return min(s i van Emde Boas tree: Operation INSERT(x Algorithm Procedure IN S E R T(x, S if S = then 3 min(s max(s x and return 4 if x < min(s then 5 swap(x, min(s 6 if x > max(s then 7 max(s x 8 i l(x 9 if S i = then 0 IN S E R T(i, P IN S E R T(r(x, S i Analysis The recursion is called at most once for every level of the tree Time complexity is O(log log U Analysis If S i =, then IN S E R T(r(x, S i only sets min(s i and max(s i A non-trivial recursion is called at most once for every level of the tree Time complexity is O(log log U Jira Fin Data Structures II 5 Jira Fin Data Structures II 6 van Emde Boas tree: Operation DELETE(x Algorithm Time complexity is also O(log log U Function DELETEMIN also returns the new minimal value. Procedure DE L E T E(x, S if S = or x = min(s = max(s then 3 S and return 4 if x = min(s then 5 min(s DE L E T EMI N(S and return 6 i l(x 7 if x = min(s i = max(s i then 8 DE L E T E(i, P 9 DE L E T E(r(x, S i 0 max(s max(s max(p Procedure DE L E T EMI N(S i min(p 3 if min(s i < max(s i then 4 return DE L E T EMI N(S i 5 S i 6 i DE L E T EMI N(P 7 return min(s i Jira Fin Data Structures II 7 Jira Fin Data Structures II 7 x-fast trie (Willard [4] First step A set S is represented using an array A such that x is stored at A[x] Non-empty positions of A are interconnected using a lined list Build a binary tree T whose leaves are cells of A Every node v of T stores the minimum and maximum of the subtree of v Operation SUCC(x If A[x] is non-empty, then return A[x] next Find the largest empty subtree v containing x and its parent p If v is a left child of p, then return p min Return p max next Complexity Space: O(U SUCC: O(log U INSERT, DELETE: O(log U x-fast trie: Improvements Second step: Speed up The largest empty subtree v containing x can be found using binary search Time complexity of SUCCis O(log log U Third step: Space reduction The number of nodes of T with non-empty subtree is at most nw Store them in a hash table (e.g. Cucoo x-fast trie Space complexity O(nw SUCC: O(log log U worst case INSERT, DELETE: O(log U expected amortized Jira Fin Data Structures II 8 Jira Fin Data Structures II 9 y-fast trie (Willard [4] y-fast trie: Operations FIND and SUCC Structure Split the ordered set S into Θ(n/B blocs S i of size Θ(B where B = log U For every bloc S i choose a representative r i and let R be the set of all representatives Create x-fast trie for representatives R, called a primary tree For every bloc S i, create a balanced search tree T i, called bloc trees Space complexity Every elements of S is stored in one node of one bloc tree Space complexity of al bloc trees is O(n Space complexity of the primary tree is O( R w = O(n since R = Θ(n/ log U and w = Θ(log U Operation FIND(x Find the predecessor r i and the successor r j representatives of x in the primary tree Element x lives in T i or T j, so search both bloc trees Time complexity Representative are found in time O(log log U Searching bloc trees taes O(log B = O(log log U Time complexity of operation FINDis O(log log U Operation SUCC(x If x / S, then FIND(x finds both the predecessor and the successor of x If x S, we use a lined list to find the successor Time complexity of operation SUCCis O(log log U Jira Fin Data Structures II 30 Jira Fin Data Structures II 3

6 If x is a representative, we search only one tree. y-fast trie: Operations INSERT and DELETE Without balancing sizes of blocs Find the proper bloc tree: O(log log U time Update the bloc tree: O(log B = O(log log U time However, the size of blocs must be O(log B and number of blocs must be O(n/ log B Balancing sizes of blocs If a bloc is too large, split it If a bloc is too small, merge it with a sibling or move Ω(B elements from a sibling Updates blocs trees taes O(B time Updates the primary tree taes O(log U time Homewor: Create rules which ensures that balancing occur once every Ω(B updates Amortized cost of balancing is O( Time complexity is O(log log U expected amortized Jira Fin Data Structures II 3 Jira Fin Data Structures II 3 Remind that x-fast trie uses Cucoo hashing Representation of a vector Consider b-bits integers x 0,..., x n Every integers is prepended by 0 and concatenated, i.e. a vector is represented as n i=0 xii(b+ Using linear compression theorem, it suffices to assume that nb = O(w Operation Read(x,i Returns the i-th integers from a vector x Read(x,i = x >> i(b+ & b where b is a binary number with b ones, i.e. b+ Operation Read(x,i,a Write b into the i-th position of x Write(x,i,a = (x&m i (a << i(b+ where M i = (b+(n i 0 b+ (b+i is a mas cleaning the i-th position Jira Fin Data Structures II 3 Jira Fin Data Structures II 33 Replicate(a Sum(x Create n copies of a Replicate(a = a (0 b Return the sum n i=0 xi Sum(x = x mod b+ x = n i=0 xib+ n i=0 xi mod b+ Ran(x,a The number of elements of x smaller than a Ran(x,a = Sum(x,Cmp(Replicate(a Cmp(x,y return z such that z i = if x i y i and 0 otherwise in the difference (x i (0y i, the first bit is if and only if x i y i Cmp(x,y = (((x (0 b n y >> b&(0 b n Min(x,y m = Cmp(x, y b Insert(x,a Insert a into a sorted vector x i = Ran(x,a is the possition for a in x Filter and move x i,..., x n to left m filters positions of x where x i y i Min(x,y = (x& m (y&m Jira Fin Data Structures II 34 Jira Fin Data Structures II 35 assuming w Ω(b Weight(a Unpac(a Create a vector x such that x i equals to the i-th bit a i of a y = Replicate(a &( b,...,, i.e. y i = i a i Unpac(a = Cmp(y, (0 b n Unpac π(a according to to a permutation π The Hamming weight of a Weight(a = Sum(Unpac(a Permute π (a Permute bits in a according to a permutation π Permute π(a = Pac(Unpac π(a Pac(a Consider that b is decreased by one and apply Sum Since the vector is not properly formatted, modulo cannot be used Pac(a = ((a (0 b n >> bn& b LSB(a The least significant bit in a LSB(a = Weight(a (a - MSB(a MSB(a = b--lsb(permute revers(a Jira Fin Data Structures II 36 Jira Fin Data Structures II 37

7 Fusion trees (Fredman, Willard, 993 [] MSB(a on O(w bits b = w Split a into l bloc of b bits and padding x = (a&(0 b l ((a&(0 b l >> b Now, padding is zero and x i = 0 if and only if i-th bloc is zero y = Cmp(x, y i = if if and only if i-th bloc is non-zero j = MSB(Pac(y j is the first non-empty bloc z = (a >> (b + j& b+ z is the content of the j-th bloc MSB(a = (b + j+ MSB(z Fusion node Let = O (w /5 A fusion node stores eys x 0,..., x A fusion node can find Ran, SUCC and PRED in O( Construction time is Poly( Fusion tree B-tree for B = Nodes are Fusion nodes The depth of the tree is O(log B (n = O ( log n log w Time complexity of Ran, SUCC and PRED is O ( log n log w Jira Fin Data Structures II 38 Jira Fin Data Structures II 39 Fusion trees Fusion node Splitting node is a node of degree. We call them splitting levels. 3 Note that every level of a BST correspond to one bit in the binary code. Bits corresponding to splitting levels are called splitting bits. Since only splitting bits are significant, s(x select these splitting bits from x. Consider a binary search tree (BST on eys x 0,..., x The number of leaves is, so the number of splitting nodes is Hence, at most levels has a splitting node Let s(x select all splitting bits from the binary code of x 3 Observe that s(x 0 < s(x < < s(x S = (s(x 0,..., s(x The number of bits of S is at most = O (w /5 so S can be stored in a single w-word Ran(x can be computed using Ran(s(x,S in O( However, we need to compute s(x in O( We find a(x of length O ( 4 containing s(x with extra zeros on the same position of all x Jira Fin Data Structures II 40 Jira Fin Data Structures II 40 Fusion trees For every b 0 < < b there exist m 0 < < m such that b i + m j are pair-wise different for all i and j b 0 + m 0 < < b + m (b + m (b 0 + m 0 = O ( 4 Splitting bits with extra zeros y = x& i b i filters all splitting bits j xb i b i +m j z = y j m j = i There is no carry since b i + m j are pair-wise different a(x = z&( i b i +m i >> (b 0 + m 0 Jira Fin Data Structures II 4 Jira Fin Data Structures II 4 Jira Fin Data Structures II 43 Jira Fin Data Structures II 44

8 Jira Fin Data Structures II 45 Jira Fin Data Structures II 46 [] Michael L Fredman and Dan E Willard. Surpassing the information theoretic bound with fusion trees. Journal of computer and system sciences, 47(3:44 436, 993. [] Torben Hagerup, Peter Bro Miltersen, and Rasmus Pagh. Deterministic dictionaries. Journal of Algorithms, 4(:69 85, 00. [3] Peter van Emde Boas. Preserving order in a forest in less than logarithmic time. In 6th Annual Symposium on Foundations of Computer Science (sfcs 975, pages IEEE, 975. [4] Dan E Willard. Log-logarithmic worst-case range queries are possible in space θ (n. Information Processing Letters, 7(:8 84, 983. Jira Fin Data Structures II 47 Jira Fin Data Structures II 47