Tabulation-Based Hashing
|
|
|
- Esmond Beasley
- 6 years ago
- Views:
Transcription
1 Mihai Pǎtraşcu Mikkel Thorup April 23, 2010
2 Applications of Hashing Hash tables: chaining x a t v f s r
3 Applications of Hashing Hash tables: chaining linear probing m c a f t x y t
4 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing a s z y f x w r b
5 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing a s z y f x w r b
6 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing a s z y f x w r b
7 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing a s z y f x w r b
8 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing a s z y f x w r b
9 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing Sketching and streaming: moment estimation: F 2 ( x) = i x2 i
10 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing Sketching and streaming: moment estimation: F 2 ( x) = i x2 i sketch A and B to later find A B A B
11 Applications of Hashing Hash tables: chaining linear probing cuckoo hashing Sketching and streaming: moment estimation: F 2 ( x) = i x2 i sketch A and B to later find A B A B etc, etc.
12 Minwise independence Hash each set through h, keen the minimum A B A B = Pr [min h(a) = min h(b)] h repeat with k different h; keep smallest k items with one h
13 Minwise independence Hash each set through h, keen the minimum A B A B = Pr [min h(a) = min h(b)] h repeat with k different h; keep smallest k items with one h The guarantee we need on h: minwise independence ( )S, x : Pr[x < min h(s)] = 1 S +1
14 Minwise independence Hash each set through h, keen the minimum A B A B = Pr [min h(a) = min h(b)] h repeat with k different h; keep smallest k items with one h The guarantee we need on h: minwise independence ( )S, x : Pr[x < min h(s)] = 1 S +1 Not feasible... Approximate: ( )S, x : Pr[x < min h(s)] = 1±ε S +1 Approximation = ε + f ( # repetitions )
15 Minwise independence Hash each set through h, keen the minimum A B A B = Pr [min h(a) = min h(b)] h repeat with k different h; keep smallest k items with one h The guarantee we need on h: minwise independence ( )S, x : Pr[x < min h(s)] = 1 S +1 Not feasible... Approximate: ( )S, x : Pr[x < min h(s)] = 1±ε S +1 Approximation = ε + f ( # repetitions ) NB: for weighted A, B the generalization is priority sampling
16 Carter & Wegman (1977) A family H = {h : [u] [b]} is k-independent iff: ( )x u, h(x) is uniform in [b]; ( )x 1,..., x k [u], h(x 1 ),..., h(x k ) are independent.
17 Carter & Wegman (1977) A family H = {h : [u] [b]} is k-independent iff: ( )x u, h(x) is uniform in [b]; ( )x 1,..., x k [u], h(x 1 ),..., h(x k ) are independent. Prototypical example: degree k polynomial u prime; choose a 0, a 1,..., a k 1 randomly in [u]; h(x) = ( a 0 + a 1 x + + a k 1 x k 1) mod b.
18 How much independence? Chaining 2 Linear probing 5 [Pagh 2, Ružić 07] 5 [PT 10] Cuckoo hashing O(lg n) 6 [Cohen, Kane 05] F 2 estimation 4 [Thorup, Zhang 04] ε-minwise indep. O(lg 1 ε ) [Indyk 99] Ω(lg 1 ε ) [PT 10]
19 How much independence? Chaining 2 Linear probing 5 [Pagh 2, Ružić 07] 5 [PT 10] Cuckoo hashing O(lg n) 6 [Cohen, Kane 05] F 2 estimation 4 [Thorup, Zhang 04] ε-minwise indep. O(lg 1 ε ) [Indyk 99] Ω(lg 1 ε ) [PT 10] Chaining: time = #{x h(x) = h(query)} E[time] = n Pr[h(x) = h(query)] = n 1 b = O(1)
20 How much independence? Chaining 2 Linear probing 5 [Pagh 2, Ružić 07] 5 [PT 10] Cuckoo hashing O(lg n) 6 [Cohen, Kane 05] F 2 estimation 4 [Thorup, Zhang 04] ε-minwise indep. O(lg 1 ε ) [Indyk 99] Ω(lg 1 ε ) [PT 10] Chaining: time = #{x h(x) = h(query)} E[time] = n Pr[h(x) = h(query)] = n 1 b = O(1) Cuckoo hashing: components in random graphs have size O(lg n)
21 How much independence? Chaining 2 Linear probing 5 [Pagh 2, Ružić 07] 5 [PT 10] Cuckoo hashing O(lg n) 6 [Cohen, Kane 05] F 2 estimation 4 [Thorup, Zhang 04] ε-minwise indep. O(lg 1 ε ) [Indyk 99] Ω(lg 1 ε ) [PT 10] Chaining: time = #{x h(x) = h(query)} E[time] = n Pr[h(x) = h(query)] = n 1 b = O(1) Cuckoo hashing: components in random graphs have size O(lg n) Minwise independence: k-level inclusion/exclusion estimates probabilities to ±2 k.
22 Linear probing
23 Implementing k-independence Goals: constant time for ω(1) independence practical solution?
24 Implementing k-independence Goals: constant time for ω(1) independence practical solution? Lower bound [Siegel 90s]: With space u 1/q, query time min{k, q}.
25 Implementing k-independence Goals: constant time for ω(1) independence practical solution? Lower bound [Siegel 90s]: With space u 1/q, query time min{k, q}. Tabulation hashing: q basic characters: x (x 1,..., x q ) d derived characters: y i = f i (x 1,..., x q ) store q + d random tables T i [ u 1/q ] h(x) = T 1 [q 1 ] T q [x q ] T q+1 [y 1 ]
26 Independence # characters [Carter, Wegman 77] 3 q ( ) [Siegel 90s] n Ω(1) q O(q) [Dietzf., Woelfel 03] k k q [Thorup, Zhang 04] k (k 1)(q 1) [Thorup, Zhang 10] 5 2q 1 recent ω(1) O(q 2 ) ( ) simple tabulation (no derived characters)
27 Peeling (q = 2, k = 3) (x 1, x 2 ) T 1 [x 1 ] T 2 [x 2 ] Let s prove independence of {a, b, c}.
28 Peeling (q = 2, k = 3) (x 1, x 2 ) T 1 [x 1 ] T 2 [x 2 ] Let s prove independence of {a, b, c}. a 1 a 2 Peeling: b 1 b 2 If a i is unique (a i b i, c i ) c 1 c 2 = h(a) independent of h(b), h(c)
29 Peeling (q = 2, k = 3) (x 1, x 2 ) T 1 [x 1 ] T 2 [x 2 ] Let s prove independence of {a, b, c}. a 1 a 2 Peeling: b 1 b 2 If a i is unique (a i b i, c i ) c 1 c 2 = h(a) independent of h(b), h(c) Any set of 3 keys is peelable, thus independent.
30 Peeling (q = 2, k = 4) (x 1, x 2 ) T 1 [x 1 ] T 2 [x 2 ] T 3 [x 1 + x 2 ] Let s prove {a, b, c, d} are independent. if we can peel, reduce to 3-independence. the only non-peelable configuration:
31 Peeling (q = 2, k = 4) (x 1, x 2 ) T 1 [x 1 ] T 2 [x 2 ] T 3 [x 1 + x 2 ] Let s prove {a, b, c, d} are independent. if we can peel, reduce to 3-independence. the only non-peelable configuration: x s x + s x t x + t y s y + s y t y + t
32 Peeling (q = 2, k = 4) (x 1, x 2 ) T 1 [x 1 ] T 2 [x 2 ] T 3 [x 1 + x 2 ] Let s prove {a, b, c, d} are independent. if we can peel, reduce to 3-independence. the only non-peelable configuration: x s x + s x t x + t y s y + s y t y + t Only possible equalities: x + s = y + t or x + t = y + s. Both cannot hold, so we have peeling in derived character.
33 5-independence [PT 10] Theorem: Any 4-independent tabulation is 5-independent! Among any 5 keys, one is independent in the basic characters.
34 5-independence [PT 10] Theorem: Any 4-independent tabulation is 5-independent! Among any 5 keys, one is independent in the basic characters. any unique character peel
35 5-independence [PT 10] Theorem: Any 4-independent tabulation is 5-independent! Among any 5 keys, one is independent in the basic characters. any unique character peel otherwise, any dimension looks like: three 0, two 1
36 5-independence [PT 10] Theorem: Any 4-independent tabulation is 5-independent! Among any 5 keys, one is independent in the basic characters. any unique character peel otherwise, any dimension looks like: three 0, two 1 two columns have Hamming distance = 4 a b c d e
37 5-independence [PT 10] Theorem: Any 4-independent tabulation is 5-independent! Among any 5 keys, one is independent in the basic characters. any unique character peel otherwise, any dimension looks like: three 0, two 1 two columns have Hamming distance = 4 a b c d e all columns at Hamming distance = 2 a b c d e NB: h(a) = h(b) h(c) h(d) If e independent of b, c, d, also independent of f(b, c, d).
38 Putting it together Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 [Thorup, Zhang 04] 4 2q 1 linear probing q 1 ε-minwise Θ(lg 1 ) [Thorup, Zhang 04] k (k 1)(q 1) ε cuckoo hashing O(lg n)? [Siegel 90s] n Ω(1) q O(q)
39 Putting it together Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 [Thorup, Zhang 04] 4 2q 1 linear probing q 1 ε-minwise Θ(lg 1 ) [Thorup, Zhang 04] k (k 1)(q 1) ε cuckoo hashing O(lg n)? [Siegel 90s] n Ω(1) q O(q) What exactly are we doing here?
40 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe...
41 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe... Simple tabulation:
42 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe... Simple tabulation:
43 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe... Simple tabulation: preserves 4 th moment bound F 2 estimation
44 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe... Simple tabulation: preserves 4 th moment bound F 2 estimation 1-in-5 indep. linear probing in expected O(1) time
45 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe... Simple tabulation: preserves 4 th moment bound F 2 estimation 1-in-5 indep. linear probing in expected O(1) time minwise independence with ε = ε(n) = o(1).
46 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe... Simple tabulation: preserves 4 th moment bound F 2 estimation 1-in-5 indep. linear probing in expected O(1) time minwise independence with ε = ε(n) = o(1). Chernoff concentration O(lg n) query time w.h.p.
47 The Power of Simple Tabulation Algorithm Indep. Scheme Indep. Characters chaining 2 simple tabulation 3 q F 2 estimation 4 simple tabulation 3 q linear probing 5 simple tabulation 3 q ε-minwise Θ(lg 1 ) simple tabulation 3 q ε cuckoo hashing O(lg n)? maybe... Simple tabulation: preserves 4 th moment bound F 2 estimation 1-in-5 indep. linear probing in expected O(1) time minwise independence with ε = ε(n) = o(1). Chernoff concentration O(lg n) query time w.h.p. preserve moments in linear probing, chaining: F p w/ simple tabulation = F p w/ truly random + o(1)
48 Simple tabulation as a PRG Pseudorandom numbers h(0), h(1), h(1),... f(0) f(1)... f(s 1) g(0) h(0) h(1)... h(s 1) g(1) h(s) h(s + 1)... h(2s 1) g(2) h(2s) h(2s + 1)... h(3s 1)...
49 Simple tabulation as a PRG Pseudorandom numbers h(0), h(1), h(1),... f(0) f(1)... f(s 1) g(0) h(0) h(1)... h(s 1) g(1) h(s) h(s + 1)... h(2s 1) g(2) h(2s) h(2s + 1)... h(3s 1)... Use S = O(lg n) truly random numbers. Compute lg n independent g( ), but rarely. PRG has: concentration (load balancing,... ) minwise independence (treaps,... )
50 Wait, there s more! What more can we ask for?
51 Wait, there s more! What more can we ask for? minwise independence with ε = ε(n) = o(1) minwise independence with ε = ε(u) = o(1)
52 Wait, there s more! What more can we ask for? minwise independence with ε = ε(n) = o(1) minwise independence with ε = ε(u) = o(1) linear probing/chaining O(1) exp. time, O(lg n) w.h.p. for k lg n, any k operations work in O(k) time w.h.p.
53 Wait, there s more! What more can we ask for? minwise independence with ε = ε(n) = o(1) minwise independence with ε = ε(u) = o(1) Experiments: {0, 1} q is counterexample? linear probing/chaining O(1) exp. time, O(lg n) w.h.p. for k lg n, any k operations work in O(k) time w.h.p.
54 Wait, there s more! What more can we ask for? minwise independence with ε = ε(n) = o(1) minwise independence with ε = ε(u) = o(1) Experiments: {0, 1} q is counterexample? linear probing/chaining O(1) exp. time, O(lg n) w.h.p. for k lg n, any k operations work in O(k) time w.h.p. We only get k = n ε for any ε > 0. Counterexample for n o(1).
55 Wait, there s more! What more can we ask for? minwise independence with ε = ε(n) = o(1) minwise independence with ε = ε(u) = o(1) Experiments: {0, 1} q is counterexample? linear probing/chaining O(1) exp. time, O(lg n) w.h.p. for k lg n, any k operations work in O(k) time w.h.p. We only get k = n ε for any ε > 0. Counterexample for n o(1). Simple++: h 1 : [u] [b], h 2 : [u] [u 1/q ]. Just simple tabulation... h(x) = h 1 (x) T [h 2 (x)]. All previous properties, plus: minwise independence with ε(u) = o(1). linear probing/chaining with buffer k = O(lg n).
56 T HE EN D
Bottom-k and Priority Sampling, Set Similarity and Subset Sums with Minimal Independence. Mikkel Thorup University of Copenhagen
Bottom-k and Priority Sampling, Set Similarity and Subset Sums with Minimal Independence Mikkel Thorup University of Copenhagen Min-wise hashing [Broder, 98, Alta Vita] Jaccard similary of sets A and B
Lecture Lecture 3 Tuesday Sep 09, 2014
CS 4: Advanced Algorithms Fall 04 Lecture Lecture 3 Tuesday Sep 09, 04 Prof. Jelani Nelson Scribe: Thibaut Horel Overview In the previous lecture we finished covering data structures for the predecessor
The Power of Simple Tabulation Hashing
The Power of Simple Tabulation Hashing Mihai Pǎtraşcu AT&T Labs Mikkel Thorup AT&T Labs Abstract Randomized algorithms are often enjoyed for their simplicity, but the hash functions used to yield the desired
The Power of Simple Tabulation Hashing
The Power of Simple Tabulation Hashing Mihai Pǎtraşcu AT&T Labs Florham Park, NJ mip@alum.mit.edu Mikkel Thorup AT&T Labs Florham Park, NJ mthorup@research.att.com ABSTRACT Randomized algorithms are often
Basics of hashing: k-independence and applications
Basics of hashing: k-independence and applications Rasmus Pagh Supported by: 1 Agenda Load balancing using hashing! - Analysis using bounded independence! Implementation of small independence! Case studies:!
arxiv: v3 [cs.ds] 11 May 2017
![arxiv: v3 [cs.ds] 11 May 2017](/thumbs/82/85614797.jpg "arxiv: v3 [cs.ds] 11 May 2017")
Lecture Notes on Linear Probing with 5-Independent Hashing arxiv:1509.04549v3 [cs.ds] 11 May 2017 Mikkel Thorup May 12, 2017 Abstract These lecture notes show that linear probing takes expected constant
UNIFORM HASHING IN CONSTANT TIME AND OPTIMAL SPACE
UNIFORM HASHING IN CONSTANT TIME AND OPTIMAL SPACE ANNA PAGH AND RASMUS PAGH Abstract. Many algorithms and data structures employing hashing have been analyzed under the uniform hashing assumption, i.e.,
CS 473: Algorithms. Ruta Mehta. Spring University of Illinois, Urbana-Champaign. Ruta (UIUC) CS473 1 Spring / 32
CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 32 CS 473: Algorithms, Spring 2018 Universal Hashing Lecture 10 Feb 15, 2018 Most
arxiv: v5 [cs.ds] 3 Jan 2017
![arxiv: v5 [cs.ds] 3 Jan 2017](/thumbs/79/80251273.jpg "arxiv: v5 [cs.ds] 3 Jan 2017")
Fast and Powerful Hashing using Tabulation Mikkel Thorup University of Copenhagen January 4, 2017 arxiv:1505.01523v5 [cs.ds] 3 Jan 2017 Abstract Randomized algorithms are often enjoyed for their simplicity,
Lecture 4 Thursday Sep 11, 2014
CS 224: Advanced Algorithms Fall 2014 Lecture 4 Thursday Sep 11, 2014 Prof. Jelani Nelson Scribe: Marco Gentili 1 Overview Today we re going to talk about: 1. linear probing (show with 5-wise independence)
arxiv: v2 [cs.ds] 6 Dec 2013
![arxiv: v2 [cs.ds] 6 Dec 2013](/thumbs/88/115708212.jpg "arxiv: v2 [cs.ds] 6 Dec 2013")
Simple Tabulation, Fast Expanders, Double Tabulation, and High Independence arxiv:1311.3121v2 [cs.ds] 6 Dec 2013 Mikkel Thorup University of Copenhagen mikkel2thorup@gmail.com December 9, 2013 Abstract
On Randomness in Hash Functions. Martin Dietzfelbinger Technische Universität Ilmenau
On Randomness in Hash Functions Martin Dietzfelbinger Technische Universität Ilmenau Martin Dietzfelbinger STACS Paris, March 2, 2012 Co-Workers Andrea Montanari Andreas Goerdt Christoph Weidling Martin
Some notes on streaming algorithms continued
U.C. Berkeley CS170: Algorithms Handout LN-11-9 Christos Papadimitriou & Luca Trevisan November 9, 016 Some notes on streaming algorithms continued Today we complete our quick review of streaming algorithms.
Hash Tables. Given a set of possible keys U, such that U = u and a table of m entries, a Hash function h is a
Hash Tables Given a set of possible keys U, such that U = u and a table of m entries, a Hash function h is a mapping from U to M = {1,..., m}. A collision occurs when two hashed elements have h(x) =h(y).
CSE 190, Great ideas in algorithms: Pairwise independent hash functions
CSE 190, Great ideas in algorithms: Pairwise independent hash functions 1 Hash functions The goal of hash functions is to map elements from a large domain to a small one. Typically, to obtain the required
CS 591, Lecture 6 Data Analytics: Theory and Applications Boston University
CS 591, Lecture 6 Data Analytics: Theory and Applications Boston University Babis Tsourakakis February 8th, 2017 Universal hash family Notation: Universe U = {0,..., u 1}, index space M = {0,..., m 1},
Power of d Choices with Simple Tabulation
Power of d Choices with Simple Tabulation Anders Aamand 1 BARC, University of Copenhagen, Universitetsparken 1, Copenhagen, Denmark. aa@di.ku.dk https://orcid.org/0000-0002-0402-0514 Mathias Bæk Tejs Knudsen
Hashing. Martin Babka. January 12, 2011
Hashing Martin Babka January 12, 2011 Hashing Hashing, Universal hashing, Perfect hashing Input data is uniformly distributed. A dynamic set is stored. Universal hashing Randomised algorithm uniform choice
Cache-Oblivious Hashing
Cache-Oblivious Hashing Zhewei Wei Hong Kong University of Science & Technology Joint work with Rasmus Pagh, Ke Yi and Qin Zhang Dictionary Problem Store a subset S of the Universe U. Lookup: Does x belong
Lecture 15 April 9, 2007
6.851: Advanced Data Structures Spring 2007 Mihai Pătraşcu Lecture 15 April 9, 2007 Scribe: Ivaylo Riskov 1 Overview In the last lecture we considered the problem of finding the predecessor in its static
Power of d Choices with Simple Tabulation
Power of d Choices with Simple Tabulation Anders Aamand, Mathias Bæk Tejs Knudsen, and Mikkel Thorup Abstract arxiv:1804.09684v1 [cs.ds] 25 Apr 2018 Suppose that we are to place m balls into n bins sequentially
Lecture 2. Frequency problems
1 / 43 Lecture 2. Frequency problems Ricard Gavaldà MIRI Seminar on Data Streams, Spring 2015 Contents 2 / 43 1 Frequency problems in data streams 2 Approximating inner product 3 Computing frequency moments
Balanced Allocation Through Random Walk
Balanced Allocation Through Random Walk Alan Frieze Samantha Petti November 25, 2017 Abstract We consider the allocation problem in which m (1 ε)dn items are to be allocated to n bins with capacity d.
Cuckoo Hashing with a Stash: Alternative Analysis, Simple Hash Functions
1 / 29 Cuckoo Hashing with a Stash: Alternative Analysis, Simple Hash Functions Martin Aumüller, Martin Dietzfelbinger Technische Universität Ilmenau 2 / 29 Cuckoo Hashing Maintain a dynamic dictionary
Hash tables. Hash tables
Dictionary Definition A dictionary is a data-structure that stores a set of elements where each element has a unique key, and supports the following operations: Search(S, k) Return the element whose key
Balls and Bins: Smaller Hash Families and Faster Evaluation
Balls and Bins: Smaller Hash Families and Faster Evaluation L. Elisa Celis Omer Reingold Gil Segev Udi Wieder May 1, 2013 Abstract A fundamental fact in the analysis of randomized algorithms is that when
Hash tables. Hash tables
Dictionary Definition A dictionary is a data-structure that stores a set of elements where each element has a unique key, and supports the following operations: Search(S, k) Return the element whose key
Tight Bounds for Sliding Bloom Filters
Tight Bounds for Sliding Bloom Filters Moni Naor Eylon Yogev November 7, 2013 Abstract A Bloom filter is a method for reducing the space (memory) required for representing a set by allowing a small error
Balls and Bins: Smaller Hash Families and Faster Evaluation
Balls and Bins: Smaller Hash Families and Faster Evaluation L. Elisa Celis Omer Reingold Gil Segev Udi Wieder April 22, 2011 Abstract A fundamental fact in the analysis of randomized algorithm is that
Randomized Algorithms. Zhou Jun
Randomized Algorithms Zhou Jun 1 Content 13.1 Contention Resolution 13.2 Global Minimum Cut 13.3 *Random Variables and Expectation 13.4 Randomized Approximation Algorithm for MAX 3- SAT 13.6 Hashing 13.7
Towards Polynomial Lower Bounds for Dynamic Problems
Towards Polynomial Lower Bounds for Dynamic Problems Mihai Pǎtraşcu AT&T Labs ABSTRACT We consider a number of dynamic problems with no known poly-logarithmic upper bounds, and show that they require n
Why Simple Hash Functions Work: Exploiting the Entropy in a Data Stream
Why Simple Hash Functions Work: Exploiting the Entropy in a Data Stream ichael itzenmacher Salil Vadhan School of Engineering & Applied Sciences Harvard University Cambridge, A 02138 {michaelm,salil}@eecs.harvard.edu
CS 580: Algorithm Design and Analysis
CS 580: Algorithm Design and Analysis Jeremiah Blocki Purdue University Spring 2018 Announcements: Homework 6 deadline extended to April 24 th at 11:59 PM Course Evaluation Survey: Live until 4/29/2018
Computing the Entropy of a Stream
Computing the Entropy of a Stream To appear in SODA 2007 Graham Cormode graham@research.att.com Amit Chakrabarti Dartmouth College Andrew McGregor U. Penn / UCSD Outline Introduction Entropy Upper Bound
Why Simple Hash Functions Work: Exploiting the Entropy in a Data Stream
Why Simple Hash Functions Work: Exploiting the Entropy in a Data Stream Michael Mitzenmacher Salil Vadhan Abstract Hashing is fundamental to many algorithms and data structures widely used in practice.
12 Hash Tables Introduction Chaining. Lecture 12: Hash Tables [Fa 10]
![12 Hash Tables Introduction Chaining. Lecture 12: Hash Tables [Fa 10]](/thumbs/82/85322860.jpg "12 Hash Tables Introduction Chaining. Lecture 12: Hash Tables [Fa 10]")
Calvin: There! I finished our secret code! Hobbes: Let s see. Calvin: I assigned each letter a totally random number, so the code will be hard to crack. For letter A, you write 3,004,572,688. B is 28,731,569½.
Introduction to Hash Tables
Introduction to Hash Tables Hash Functions A hash table represents a simple but efficient way of storing, finding, and removing elements. In general, a hash table is represented by an array of cells. In
A Unifying Framework for l 0 -Sampling Algorithms
Distributed and Parallel Databases manuscript No. (will be inserted by the editor) A Unifying Framework for l 0 -Sampling Algorithms Graham Cormode Donatella Firmani Received: date / Accepted: date Abstract
Explicit and Efficient Hash Functions Suffice for Cuckoo Hashing with a Stash
Explicit and Efficient Hash Functions Suffice for Cuckoo Hashing with a Stash Martin Aumu ller, Martin Dietzfelbinger, Philipp Woelfel? Technische Universita t Ilmenau,? University of Calgary ESA Ljubljana,
Lecture 4 February 2nd, 2017
CS 224: Advanced Algorithms Spring 2017 Prof. Jelani Nelson Lecture 4 February 2nd, 2017 Scribe: Rohil Prasad 1 Overview In the last lecture we covered topics in hashing, including load balancing, k-wise
A Tight Lower Bound for Dynamic Membership in the External Memory Model
A Tight Lower Bound for Dynamic Membership in the External Memory Model Elad Verbin ITCS, Tsinghua University Qin Zhang Hong Kong University of Science & Technology April 2010 1-1 The computational model
CS5314 Randomized Algorithms. Lecture 15: Balls, Bins, Random Graphs (Hashing)
CS5314 Randomized Algorithms Lecture 15: Balls, Bins, Random Graphs (Hashing) 1 Objectives Study various hashing schemes Apply balls-and-bins model to analyze their performances 2 Chain Hashing Suppose
1 Maintaining a Dictionary
15-451/651: Design & Analysis of Algorithms February 1, 2016 Lecture #7: Hashing last changed: January 29, 2016 Hashing is a great practical tool, with an interesting and subtle theory too. In addition
compare to comparison and pointer based sorting, binary trees
Admin Hashing Dictionaries Model Operations. makeset, insert, delete, find keys are integers in M = {1,..., m} (so assume machine word size, or unit time, is log m) can store in array of size M using power:
Lower Bounds for Dynamic Connectivity (2004; Pǎtraşcu, Demaine)
Lower Bounds for Dynamic Connectivity (2004; Pǎtraşcu, Demaine) Mihai Pǎtraşcu, MIT, web.mit.edu/ mip/www/ Index terms: partial-sums problem, prefix sums, dynamic lower bounds Synonyms: dynamic trees 1
Hash tables. Hash tables
Basic Probability Theory Two events A, B are independent if Conditional probability: Pr[A B] = Pr[A] Pr[B] Pr[A B] = Pr[A B] Pr[B] The expectation of a (discrete) random variable X is E[X ] = k k Pr[X
CS369N: Beyond Worst-Case Analysis Lecture #6: Pseudorandom Data and Universal Hashing
CS369N: Beyond Worst-Case Analysis Lecture #6: Pseudorandom Data and Universal Hashing Tim Roughgarden April 4, 204 otivation: Linear Probing and Universal Hashing This lecture discusses a very neat paper
6.1 Occupancy Problem
15-859(M): Randomized Algorithms Lecturer: Anupam Gupta Topic: Occupancy Problems and Hashing Date: Sep 9 Scribe: Runting Shi 6.1 Occupancy Problem Bins and Balls Throw n balls into n bins at random. 1.
Advanced Data Structures
Simon Gog gog@kit.edu - 0 Simon Gog: KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association www.kit.edu Dynamic Perfect Hashing What we want: O(1) lookup
BEYOND POST QUANTUM CRYPTOGRAPHY
BEYOND POST QUANTUM CRYPTOGRAPHY Mark Zhandry Stanford University Joint work with Dan Boneh Classical Cryptography Post-Quantum Cryptography All communication stays classical Beyond Post-Quantum Cryptography
? 11.5 Perfect hashing. Exercises
11.5 Perfect hashing 77 Exercises 11.4-1 Consider inserting the keys 10; ; 31; 4; 15; 8; 17; 88; 59 into a hash table of length m 11 using open addressing with the auxiliary hash function h 0.k/ k. Illustrate
Data Stream Methods. Graham Cormode S. Muthukrishnan
Data Stream Methods Graham Cormode graham@dimacs.rutgers.edu S. Muthukrishnan muthu@cs.rutgers.edu Plan of attack Frequent Items / Heavy Hitters Counting Distinct Elements Clustering items in Streams Motivating
Lecture 10. Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1
Lecture 10 Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1 Recap Sublinear time algorithms Deterministic + exact: binary search Deterministic + inexact: estimating diameter in a
Data Streams & Communication Complexity
Data Streams & Communication Complexity Lecture 1: Simple Stream Statistics in Small Space Andrew McGregor, UMass Amherst 1/25 Data Stream Model Stream: m elements from universe of size n, e.g., x 1, x
B490 Mining the Big Data
B490 Mining the Big Data 1 Finding Similar Items Qin Zhang 1-1 Motivations Finding similar documents/webpages/images (Approximate) mirror sites. Application: Don t want to show both when Google. 2-1 Motivations
Higher Lower Bounds for Near-Neighbor and Further Rich Problems
Higher Lower Bounds for Near-Neighbor and Further Rich Problems Mihai Pǎtraşcu mip@mit.edu Mikkel Thorup mthorup@research.att.com December 4, 2008 Abstract We convert cell-probe lower bounds for polynomial
Block Heavy Hitters Alexandr Andoni, Khanh Do Ba, and Piotr Indyk
Computer Science and Artificial Intelligence Laboratory Technical Report -CSAIL-TR-2008-024 May 2, 2008 Block Heavy Hitters Alexandr Andoni, Khanh Do Ba, and Piotr Indyk massachusetts institute of technology,
Nearest Neighbor Classification Using Bottom-k Sketches
Nearest Neighbor Classification Using Bottom- Setches Søren Dahlgaard, Christian Igel, Miel Thorup Department of Computer Science, University of Copenhagen AT&T Labs Abstract Bottom- setches are an alternative
1 Estimating Frequency Moments in Streams
CS 598CSC: Algorithms for Big Data Lecture date: August 28, 2014 Instructor: Chandra Chekuri Scribe: Chandra Chekuri 1 Estimating Frequency Moments in Streams A significant fraction of streaming literature
Using Hashing to Solve the Dictionary Problem (In External Memory)
Using Hashing to Solve the Dictionary Problem (In External Memory) John Iacono NYU Poly Mihai Pǎtraşcu AT&T Labs February 9, 2011 Abstract We consider the dictionary problem in external memory and improve
6.854 Advanced Algorithms
6.854 Advanced Algorithms Homework Solutions Hashing Bashing. Solution:. O(log U ) for the first level and for each of the O(n) second level functions, giving a total of O(n log U ) 2. Suppose we are using
Big Data. Big data arises in many forms: Common themes:
Big Data Big data arises in many forms: Physical Measurements: from science (physics, astronomy) Medical data: genetic sequences, detailed time series Activity data: GPS location, social network activity
Polynomial Representations of Threshold Functions and Algorithmic Applications. Joint with Josh Alman (Stanford) and Timothy M.
Polynomial Representations of Threshold Functions and Algorithmic Applications Ryan Williams Stanford Joint with Josh Alman (Stanford) and Timothy M. Chan (Waterloo) Outline The Context: Polynomial Representations,
COS433/Math 473: Cryptography. Mark Zhandry Princeton University Spring 2017
COS433/Math 473: Cryptography Mark Zhandry Princeton University Spring 2017 Last Time Hardcore Bits Hardcore Bits Let F be a one- way function with domain x, range y Definition: A function h:xà {0,1} is
Searching. Constant time access. Hash function. Use an array? Better hash function? Hash function 4/18/2013. Chapter 9
Constant time access Searching Chapter 9 Linear search Θ(n) OK Binary search Θ(log n) Better Can we achieve Θ(1) search time? CPTR 318 1 2 Use an array? Use random access on a key such as a string? Hash
Lecture 11: Hash Functions, Merkle-Damgaard, Random Oracle
CS 7880 Graduate Cryptography October 20, 2015 Lecture 11: Hash Functions, Merkle-Damgaard, Random Oracle Lecturer: Daniel Wichs Scribe: Tanay Mehta 1 Topics Covered Review Collision-Resistant Hash Functions
COS433/Math 473: Cryptography. Mark Zhandry Princeton University Spring 2017
COS433/Math 473: Cryptography Mark Zhandry Princeton University Spring 2017 Authenticated Encryption Syntax Syntax: Enc: K M à C Dec: K C à M { } Correctness: For all k K, m M, Dec(k, Enc(k,m) ) = m Unforgeability
The Cell Probe Complexity of Dynamic Range Counting
The Cell Probe Complexity of Dynamic Range Counting Kasper Green Larsen MADALGO, Department of Computer Science Aarhus University Aarhus, Denmark email: larsen@cs.au.dk August 27, 2012 Abstract In this
Hashing. Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing. Philip Bille
Hashing Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing Philip Bille Hashing Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing
DATA MINING LECTURE 6. Similarity and Distance Sketching, Locality Sensitive Hashing
DATA MINING LECTURE 6 Similarity and Distance Sketching, Locality Sensitive Hashing SIMILARITY AND DISTANCE Thanks to: Tan, Steinbach, and Kumar, Introduction to Data Mining Rajaraman and Ullman, Mining
Hashing. Hashing. Dictionaries. Dictionaries. Dictionaries Chained Hashing Universal Hashing Static Dictionaries and Perfect Hashing
Philip Bille Dictionaries Dictionary problem. Maintain a set S U = {,..., u-} supporting lookup(x): return true if x S and false otherwise. insert(x): set S = S {x} delete(x): set S = S - {x} Dictionaries
6.842 Randomness and Computation Lecture 5
6.842 Randomness and Computation 2012-02-22 Lecture 5 Lecturer: Ronitt Rubinfeld Scribe: Michael Forbes 1 Overview Today we will define the notion of a pairwise independent hash function, and discuss its
New Lower and Upper Bounds for Representing Sequences
New Lower and Upper Bounds for Representing Sequences Djamal Belazzougui 1 and Gonzalo Navarro 2 1 LIAFA, Univ. Paris Diderot - Paris 7, France. dbelaz@liafa.jussieu.fr 2 Department of Computer Science,
15-451/651: Design & Analysis of Algorithms September 13, 2018 Lecture #6: Streaming Algorithms last changed: August 30, 2018
15-451/651: Design & Analysis of Algorithms September 13, 2018 Lecture #6: Streaming Algorithms last changed: August 30, 2018 Today we ll talk about a topic that is both very old (as far as computer science
On the Optimality of the Dimensionality Reduction Method
On the Optimality of the Dimensionality Reduction Method Alexandr Andoni MIT andoni@mit.edu Piotr Indyk MIT indyk@mit.edu Mihai Pǎtraşcu MIT mip@mit.edu Abstract We investigate the optimality of (1+)-approximation
?? Compressed Matrix Multiplication
?? Compressed Matrix Multiplication RASMUS PAGH, IT University of Copenhagen We present a simple algorithm that approximates the product of n-by-n real matrices A and B. Let AB F denote the Frobenius norm
Range-efficient computation of F 0 over massive data streams
Range-efficient computation of F 0 over massive data streams A. Pavan Dept. of Computer Science Iowa State University pavan@cs.iastate.edu Srikanta Tirthapura Dept. of Elec. and Computer Engg. Iowa State
Background: Lattices and the Learning-with-Errors problem
Background: Lattices and the Learning-with-Errors problem China Summer School on Lattices and Cryptography, June 2014 Starting Point: Linear Equations Easy to solve a linear system of equations A s = b
Higher Cell Probe Lower Bounds for Evaluating Polynomials
Higher Cell Probe Lower Bounds for Evaluating Polynomials Kasper Green Larsen MADALGO, Department of Computer Science Aarhus University Aarhus, Denmark Email: larsen@cs.au.dk Abstract In this paper, we
CS6999 Probabilistic Methods in Integer Programming Randomized Rounding Andrew D. Smith April 2003
CS6999 Probabilistic Methods in Integer Programming Randomized Rounding April 2003 Overview 2 Background Randomized Rounding Handling Feasibility Derandomization Advanced Techniques Integer Programming
Introduction to Quantum Computing
Introduction to Quantum Computing Part II Emma Strubell http://cs.umaine.edu/~ema/quantum_tutorial.pdf April 13, 2011 Overview Outline Grover s Algorithm Quantum search A worked example Simon s algorithm
Lecture 6. Today we shall use graph entropy to improve the obvious lower bound on good hash functions.
CSE533: Information Theory in Computer Science September 8, 010 Lecturer: Anup Rao Lecture 6 Scribe: Lukas Svec 1 A lower bound for perfect hash functions Today we shall use graph entropy to improve the
Linear Sketches A Useful Tool in Streaming and Compressive Sensing
Linear Sketches A Useful Tool in Streaming and Compressive Sensing Qin Zhang 1-1 Linear sketch Random linear projection M : R n R k that preserves properties of any v R n with high prob. where k n. M =
2 How many distinct elements are in a stream?
Dealing with Massive Data January 31, 2011 Lecture 2: Distinct Element Counting Lecturer: Sergei Vassilvitskii Scribe:Ido Rosen & Yoonji Shin 1 Introduction We begin by defining the stream formally. Definition
Randomized Load Balancing:The Power of 2 Choices
Randomized Load Balancing: The Power of 2 Choices June 3, 2010 Balls and Bins Problem We have m balls that are thrown into n bins, the location of each ball chosen independently and uniformly at random
TTIC An Introduction to the Theory of Machine Learning. Learning from noisy data, intro to SQ model
TTIC 325 An Introduction to the Theory of Machine Learning Learning from noisy data, intro to SQ model Avrim Blum 4/25/8 Learning when there is no perfect predictor Hoeffding/Chernoff bounds: minimizing
Outline. CS38 Introduction to Algorithms. Max-3-SAT approximation algorithm 6/3/2014. randomness in algorithms. Lecture 19 June 3, 2014
Outline CS38 Introduction to Algorithms Lecture 19 June 3, 2014 randomness in algorithms max-3-sat approximation algorithm universal hashing load balancing Course summary and review June 3, 2014 CS38 Lecture
Arthur-Merlin Streaming Complexity
Weizmann Institute of Science Joint work with Ran Raz Data Streams The data stream model is an abstraction commonly used for algorithms that process network traffic using sublinear space. A data stream
Frequency Estimators
Frequency Estimators Outline for Today Randomized Data Structures Our next approach to improving performance. Count-Min Sketches A simple and powerful data structure for estimating frequencies. Count Sketches
Derandomization, Hashing and Expanders
Derandomization, Hashing and Expanders Milan Ružić A PhD Dissertation Presented to the Faculty of the IT University of Copenhagen in Partial Fulfillment of the Requirements for the PhD Degree August 2009
From Non-Adaptive to Adaptive Pseudorandom Functions
From Non-Adaptive to Adaptive Pseudorandom Functions Itay Berman Iftach Haitner January, 202 Abstract Unlike the standard notion of pseudorandom functions (PRF), a non-adaptive PRF is only required to
CS 229r: Algorithms for Big Data Fall Lecture 17 10/28
CS 229r: Algorithms for Big Data Fall 2015 Prof. Jelani Nelson Lecture 17 10/28 Scribe: Morris Yau 1 Overview In the last lecture we defined subspace embeddings a subspace embedding is a linear transformation
Lecture 5, CPA Secure Encryption from PRFs
CS 4501-6501 Topics in Cryptography 16 Feb 2018 Lecture 5, CPA Secure Encryption from PRFs Lecturer: Mohammad Mahmoody Scribe: J. Fu, D. Anderson, W. Chao, and Y. Yu 1 Review Ralling: CPA Security and
Advanced Algorithm Design: Hashing and Applications to Compact Data Representation
Advanced Algorithm Design: Hashing and Applications to Compact Data Representation Lectured by Prof. Moses Chariar Transcribed by John McSpedon Feb th, 20 Cucoo Hashing Recall from last lecture the dictionary
Algorithms for Data Science
Algorithms for Data Science CSOR W4246 Eleni Drinea Computer Science Department Columbia University Tuesday, December 1, 2015 Outline 1 Recap Balls and bins 2 On randomized algorithms 3 Saving space: hashing-based
CPSC 467: Cryptography and Computer Security
CPSC 467: Cryptography and Computer Security Michael J. Fischer Lecture 14 October 16, 2013 CPSC 467, Lecture 14 1/45 Message Digest / Cryptographic Hash Functions Hash Function Constructions Extending
DATA STREAMS. Elena Ikonomovska Jožef Stefan Institute Department of Knowledge Technologies
QUERYING AND MINING DATA STREAMS Elena Ikonomovska Jožef Stefan Institute Department of Knowledge Technologies Outline Definitions Datastream models Similarity measures Historical background Foundations
CS Data Structures and Algorithm Analysis
CS 483 - Data Structures and Algorithm Analysis Lecture VII: Chapter 6, part 2 R. Paul Wiegand George Mason University, Department of Computer Science March 22, 2006 Outline 1 Balanced Trees 2 Heaps &
COS597D: Information Theory in Computer Science October 5, Lecture 6
COS597D: Information Theory in Computer Science October 5, 2011 Lecture 6 Lecturer: Mark Braverman Scribe: Yonatan Naamad 1 A lower bound for perfect hash families. In the previous lecture, we saw that
14.1 Finding frequent elements in stream
Chapter 14 Streaming Data Model 14.1 Finding frequent elements in stream A very useful statistics for many applications is to keep track of elements that occur more frequently. It can come in many flavours