Cache-Oblivious Algorithms

Transcription

1 Cache-Oblivious Algorithms 1

2 Cache-Oblivious Model 2

3 The Unknown Machine Algorithm C program gcc Object code linux Execution Can be executed on machines with a specific class of CPUs Algorithm Java program javac Java bytecode java Interpretation Can be executed on any machine with a Java interpreter 3

4 The Unknown Machine Algorithm C program gcc Object code linux Execution Can be executed on machines with a specific class of CPUs Algorithm Java program javac Java bytecode java Interpretation Can be executed on any machine with a Java interpreter Goal Develop algorithms that are optimized w.r.t. memory hierarchies without knowing the parameters 3

5 Cache-Oblivious Model CPU Memory I/O Disk I/O model Algorithms do not know the parameters B and M Optimal off-line cache replacement strategy Frigo et al

6 Justification of the ideal-cache model Optimal replacement LRU + 2 cache size at most 2 cache misses Sleator an Tarjan, 1985 Corollary T M,B (N) = O(T 2M,B (N)) #cache misses using LRU is O(T M,B (N)) Two memory levels Optimal cache-oblivious algorithm satisfying T M,B (N) = O(T 2M,B (N)) optimal #cache misses on each level of a multilevel cache using LRU Fully associativity cache Simulation of LRU Direct mapped cache Explicit memory management Dictionary (2-universal hash functions) of cache lines in memory Expected O(1) access time to a cache line in memory 5

7 Matrix Multiplication 6

8 Matrix Multiplication Problem C = A B, c ij = k=1..n a ik b kj Layout of matrices Row major Column major 4 4-blocked Bit interleaved 7

9 Matrix Multiplication Algorithm 1: Nested loops Row major Reading a column of B uses N I/Os Total O(N 3 ) I/Os for i = 1 to N for j = 1 to N c ij = 0 for k = 1 to N c ij = c ij + a ik b kj 8

10 Matrix Multiplication Algorithm 1: Nested loops Row major Reading a column of B uses N I/Os Total O(N 3 ) I/Os for i = 1 to N for j = 1 to N c ij = 0 for k = 1 to N c ij = c ij + a ik b kj Algorithm 2: Blocked algorithm (cache-aware) s Partition A and B into blocks of size s s where s s = Θ( M) Apply Algorithm 1 to the N N matrices where s s elements are s s matrices

11 Matrix Multiplication Algorithm 1: Nested loops Row major Reading a column of B uses N I/Os Total O(N 3 ) I/Os for i = 1 to N for j = 1 to N c ij = 0 for k = 1 to N c ij = c ij + a ik b kj Algorithm 2: Blocked algorithm (cache-aware) 0 Partition A and B into blocks of size s s where s 8 s = Θ( M) Apply Algorithm 1 to the N N matrices where 32 s s 40 elements are s s matrices s s-blocked or ( row major and M = Ω(B 2 ) ) ( ( ) ) 3 O N s s 2 = O ( ) ( ) N 3 B s B = O N 3 I/Os B M s

12 Matrix Multiplication Algorithm 1: Nested loops Row major Reading a column of B uses N I/Os Total O(N 3 ) I/Os for i = 1 to N for j = 1 to N c ij = 0 for k = 1 to N c ij = c ij + a ik b kj Algorithm 2: Blocked algorithm (cache-aware) 0 Partition A and B into blocks of size s s where s 8 s = Θ( M) Apply Algorithm 1 to the N N matrices where 32 s s 40 elements are s s matrices s s-blocked or ( row major and M = Ω(B 2 ) ) ( ( ) ) 3 O N s s 2 = O ( ) ( ) N 3 B s B = O N 3 I/Os B M s Optimal Hong & Kung,

13 Matrix Multiplication Algorithm 3: Recursive algorithm (cache-oblivious) A 11 A 12 A 21 A 22 B 11 B 12 B 21 B 22 = A 11B 11 + A 12 B 21 A 11 B 12 + A 12 B 22 A 21 B 11 + A 22 B 21 A 21 B 12 + A 22 B 22 8 recursive N 2 N 2 matrix multiplications + 4 N 2 N 2 matrix sums 9

14 Matrix Multiplication Algorithm 3: Recursive algorithm (cache-oblivious) A 11 A 12 A 21 A 22 B 11 B 12 B 21 B 22 = A 11B 11 + A 12 B 21 A 11 B 12 + A 12 B 22 A 21 B 11 + A 22 B 21 A 21 B 12 + A 22 B 22 8 recursive N 2 N 2 matrix multiplications + 4 N 2 N 2 # I/Os if bit interleaved or ( row major and M = Ω(B 2 ) ) T (N) T (N) O O( N 2 ) if N ε M B 8 T ( ) ( ) N + O N 2 otherwise ( N 3 2 B M ) B matrix sums 9

15 Matrix Multiplication Algorithm 3: Recursive algorithm (cache-oblivious) A 11 A 12 A 21 A 22 B 11 B 12 B 21 B 22 = A 11B 11 + A 12 B 21 A 11 B 12 + A 12 B 22 A 21 B 11 + A 22 B 21 A 21 B 12 + A 22 B 22 8 recursive N 2 N 2 matrix multiplications + 4 N 2 N 2 # I/Os if bit interleaved or ( row major and M = Ω(B 2 ) ) T (N) T (N) O O( N 2 ) if N ε M B 8 T ( ) ( ) N + O N 2 otherwise ( N 3 2 B M ) B matrix sums Optimal Hong & Kung, 1981 Non-square matrices Frigo et al.,

16 Matrix Multiplication Algorithm 4: Strassen s algorithm (cache-oblivious) 7 recursive N 2 N 2 C 11 C 12 C 21 C 22 matrix multiplications + O(1) matrix sums = A 11 A 12 A 21 A 22 B 11 B 12 B 21 B 22 m 1 := (a 21 + a 22 a 11 )(b 22 b 12 + b 11 ) c 11 := m 2 + m 3 m 2 := a 11 b 11 c 12 := m 1 + m 2 + m 5 + m 6 m 3 := a 12 b 21 c 21 := m 1 + m 2 + m 4 m 7 m 4 := (a 11 a 21 )(b 22 b 12 ) c 22 := m 1 + m 2 + m 4 + m 5 m 5 := (a 21 + a 22 )(b 12 b 11 ) m 6 := (a 12 a 21 + a 11 a 22 )b 22 m 7 := a 22 (b 11 + b 22 b 12 b 21 ) 10

17 Matrix Multiplication Algorithm 4: Strassen s algorithm (cache-oblivious) 7 recursive N 2 N 2 C 11 C 12 C 21 C 22 matrix multiplications + O(1) matrix sums = A 11 A 12 A 21 A 22 B 11 B 12 B 21 B 22 m 1 := (a 21 + a 22 a 11 )(b 22 b 12 + b 11 ) c 11 := m 2 + m 3 m 2 := a 11 b 11 c 12 := m 1 + m 2 + m 5 + m 6 m 3 := a 12 b 21 c 21 := m 1 + m 2 + m 4 m 7 m 4 := (a 11 a 21 )(b 22 b 12 ) c 22 := m 1 + m 2 + m 4 + m 5 m 5 := (a 21 + a 22 )(b 12 b 11 ) m 6 := (a 12 a 21 + a 11 a 22 )b 22 m 7 := a 22 (b 11 + b 22 b 12 b 21 ) # I/Os if bit interleaved or ( row major and M = Ω(B 2 ) ) O( N 2 T (N) ) if N ε M B 7 T ( ) ( ) N + O N 2 otherwise T (N) O ( N log 2 7 B M 2 ) B log

18 Cache-Oblivious Search Trees 11

19 Static Cache-Oblivious Trees Recursive memory layout van Emde Boas layout h/2 A h B 1 B k h/2 A B 1 B k Degree O(1) Searches use O(log B N) I/Os Prokop

20 Static Cache-Oblivious Trees Recursive memory layout van Emde Boas layout h/2 A h B 1 B k h/2 A B 1 B k Degree O(1) Searches use O(log B N) I/Os Range reportings use O ( log B N + k B ) I/Os Prokop

21 Static Cache-Oblivious Trees Recursive memory layout van Emde Boas layout h/2 A h B 1 B k h/2 A B 1 B k Degree O(1) Searches use O(log B N) I/Os Range reportings use O ( log B N + k B ) I/Os Best possible (log 2 e + o(1)) log B N Prokop 1999 Bender, Brodal, Fagerberg, Ge, He, Hu Iacono, López-Ortiz

22 Dynamic Cache-Oblivious Trees Embed a dynamic tree of small height into a complete tree Static van Emde Boas layout Rebuild data structure whenever N doubles of halves Search O(log B N) Range Reporting Updates ) O ( log B N + k B O ( log B N + log2 N B ) Brodal, Fagerberg, Jacob

23 Example

24 Binary Trees of Small Height New If an insertion causes non-small height then rebuild subtree at nearest ancestor with sufficient few descendents Insertions require amortized time O(log 2 N) Andersson and Lai

25 Binary Trees of Small Height For each level i there is a threshold τ i = τ L + i, such that 0 < τ L = τ 0 < τ 1 < < τ H = τ U < 1 For a node v i on level i definethe density Insertion ρ(v i ) = # nodes below v i m i where m i = # possible nodes below v i with depth at most H Insert new element If depth > H then locate neirest ancestor v i with ρ(v i ) τ i and rebuild subtree at v i to have minimum height and elements evenly distributed between left and right subtrees Andersson and Lai

26 Binary Trees of Small Height Theorem Insertions require amortized time O(log 2 N) Proof Consider two redistributions of v i After the first redistribution ρ(v i ) τ i Before second redistribution a child v i+1 of v i has ρ(v i+1 ) > τ i+1 Insertions below v i : m(v i+1 ) (τ i+1 τ i ) = m(v i+1 ) Redistribution of v i costs m(v i ), i.e. per insertion below v i Total insertion cost per element m(v i ) m(v i+1 ) 2 H i=0 2 = O(log2 N) Andersson and Lai

27 Memory Layouts of Trees DFS inorder BFS van Emde Boas (in theory best) 18

28 Searches in Pointer Based Layouts pointer bfs pointer dfs pointer veb pointer random insert pointer random layout e+06 van Emde Boas layout wins, followed by the BFS layout 19

29 Searches with Implicit Layouts implicit bfs implicit dfs implicit veb implicit in-order implicit 9-ary bfs e+06 BFS layout wins due to simplicity and caching of topmost levels van Emde Boas layout requires quite complex index computations 20

30 Implicit vs Pointer Based Layouts pointer bfs implicit bfs pointer veb implicit veb e+06 BFS layout e+06 van Emde Boas layout Implicit layouts become competitive as n grows 21

31 Insertions in Implicit Layouts 0.1 implicit bfs random inserts implicit in-order random inserts implicit veb random inserts e+06 Insertions are rather slow (factor over searches) 22

32 Summary Dynamic cache-oblivious search trees Search O(log B N) Range Reporting Updates ) O ( log B N + k B O ( log B N + log2 N B Update time O(log B N) by one level of indirection (implies sub-optimal range reporting) Importance of memory layouts van Emde Boas layout gives good cache performance Computation time is important when considering caches ) 23

33 Cache-Oblivious Sorting 24

34 Sorting Problem Input : array containing x 1,..., x N Output : array with x 1,..., x N in sorted order Elements can be compared and copied

35 Binary Merge-Sort Ouput Merging Merging Merging Merging Input

36 Binary Merge-Sort Ouput Merging Merging Merging Merging Input Recursive; two arrays; size O(M) internally in cache O(N log N) comparisons O ( N log ) B 2 N M I/Os 26

37 Merge-Sort Degree I/O 2 O ( N log ) B 2 N M d (d M 1) B Θ ( M B ) O ( N B log d N M O ( N log ) B M/B N M = O(SortM,B (N)) Aggarwal and Vitter 1988 Funnel-Sort 2 O( 1 ε Sort M,B(N)) (M B 1+ε ) Frigo, Leiserson, Prokop and Ramachandran 1999 ) Brodal and Fagerberg

38 Lower Bound Brodal and Fagerberg 2003 Block Size Memory I/Os Machine 1 B 1 M t 1 Machine 2 B 2 M t 2 One algorithm, two machines, B 1 B 2 Trade-off 8t 1 B 1 + 3t 1 B 1 log 8Mt 2 t 1 B 1 N log N M 1.45N 28

39 Lower Bound Assumption I/Os Lazy Funnel-sort Binary Merge-sort B M 1 ε (a) B 2 = M 1 ε : Sort B2,M(N) (b) B 1 = 1 : Sort B1,M(N) 1 ε B M/2 (a) B 2 = M/2 : Sort B2,M(N) (b) B 1 = 1 : Sort B1,M(N) log M Corollary (a) (b) 29

40 Funnel-Sort 30

41 k-merger Frigo et al., FOCS 99 Sorted output stream M k sorted input streams 31

42 k-merger Frigo et al., FOCS 99 Sorted output stream M 0 k 1/2 -mergers M Recursive def. = B 1 B k buffers of size k 3/2 M 1 M k k sorted input streams 31

43 k-merger Frigo et al., FOCS 99 Sorted output stream M 0 k 1/2 -mergers M Recursive def. = B 1 B k buffers of size k 3/2 M 1 M k k sorted input streams M 0 B 1 M 1 B 2 M 2 B k M k Recursive Layout 31

44 Lazy k-merger Brodal and Fagerberg 2002 M 0 B 1 B k M 1 M k 32

45 Lazy k-merger Brodal and Fagerberg 2002 M 0 B 1 B k M 1 M k Procedure Fill(v) while out-buffer not full if left in-buffer empty Fill(left child) if right in-buffer empty Fill(right child) perform one merge step 32

46 Lazy k-merger Brodal and Fagerberg 2002 M 0 B 1 B k M 1 M k Procedure Fill(v) while out-buffer not full if left in-buffer empty Fill(left child) if right in-buffer empty Fill(right child) perform one merge step Lemma If M B 2 and output buffer has size k 3 then O( k3 B log M(k 3 ) + k) I/Os are done during an invocation of Fill(root) 32

47 Funnel-Sort Brodal and Fagerberg 2002 Frigo, Leiserson, Prokop and Ramachandran 1999 Divide input in N 1/3 segments of size N 2/3 Recursively Funnel-Sort each segment Merge sorted segments by an N 1/3 -merger k N 1/3 N 2/9 N 4/

48 Funnel-Sort Brodal and Fagerberg 2002 Frigo, Leiserson, Prokop and Ramachandran 1999 Divide input in N 1/3 segments of size N 2/3 Recursively Funnel-Sort each segment Merge sorted segments by an N 1/3 -merger k N 1/3 N 2/9 N 4/27. 2 Theorem Funnel-Sort performs O(Sort M,B (N)) I/Os for M B 2 33

49 Hardware Processor type Pentium 4 Pentium 3 MIPS Workstation Dell PC Delta PC SGI Octane Operating system GNU/Linux Kernel GNU/Linux Kernel IRIX version 6.5 version version Clock rate 2400 MHz 800 MHz 175 MHz Address space 32 bit 32 bit 64 bit Integer pipeline stages L1 data cache size 8 KB 16 KB 32 KB L1 line size 128 Bytes 32 Bytes 32 Bytes L1 associativity 4 way 4 way 2 way L2 cache size 512 KB 256 KB 1024 KB L2 line size 128 Bytes 32 Bytes 32 Bytes L2 associativity 8 way 4 way 2 way TLB entries TLB associativity Full 4 way 64 way TLB miss handler Hardware Hardware Software Main memory 512 MB 256 MB 128 MB 34

50 Wall Clock 100.0µs Pentium 4, 512/512 ffunnelsort funnelsort lowscosa stdsort ami_sort msort-c msort-m Wall clock time per element 10.0µs 1.0µs 0.1µs 1,000,000 10,000, ,000,000 1,000,000,000 Elements Kristoffer Vinther

51 Page Faults Pentium 4, 512/512 ffunnelsort funnelsort lowscosa stdsort msort-c msort-m Page faults per block of elements ,000,000 10,000, ,000,000 1,000,000,000 Elements Kristoffer Vinther

52 Cache Misses MIPS 10000, 1024/128 ffunnelsort funnelsort lowscosa stdsort msort-c msort-m L2 cache misses per lines of elements ,000 1,000,000 10,000, ,000,000 1,000,000,000 Elements Kristoffer Vinther

53 TLB Misses 10.0 MIPS 10000, 1024/128 ffunnelsort funnelsort lowscosa stdsort msort-c msort-m TLB misses per block of elements ,000 1,000,000 10,000, ,000,000 1,000,000,000 Elements Kristoffer Vinther

54 Conclusions Cache oblivious sorting is possible requires a tall cache assumption M B 1+ε comparable performance with cache aware algorithms Future work more experimental justification for the cache oblivious model limitations of the model time space trade-offs? tool-box for cache oblivious algorithms 39