Big Data Analytics. Lucas Rego Drumond

Transcription

1 Lucas Rego Drumond Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany Map Reduce I Map Reduce I 1 / 32

2 Outline 1. Introduction 2. Parallel Computing 3. Parallel programming paradigms Map Reduce I 1 / 32

3 1. Introduction Outline 1. Introduction 2. Parallel Computing 3. Parallel programming paradigms Map Reduce I 1 / 32

4 1. Introduction Overview Part III Machine Learning Algorithms Part II Large Scale Computational Models Part I Distributed Database Distributed File System Map Reduce I 1 / 32

5 2. Parallel Computing Outline 1. Introduction 2. Parallel Computing 3. Parallel programming paradigms Map Reduce I 2 / 32

6 2. Parallel Computing Why do we need a Computational Model? Our data is nicely stored in a distributed infrastructure We have a number of computers at our disposal We want our analytics software to take advantage of all this computing power When programming we want to focus on understanding our data and not our infrastructure Map Reduce I 2 / 32

7 2. Parallel Computing Shared Memory Infrastructure Processor Processor Processor Memory Map Reduce I 3 / 32

8 2. Parallel Computing Distributed Infrastructure Network Processor Processor Processor Processor Memory Memory Memory Memory Map Reduce I 4 / 32

9 3. Parallel programming paradigms Outline 1. Introduction 2. Parallel Computing 3. Parallel programming paradigms Map Reduce I 5 / 32

10 3. Parallel programming paradigms Parallel Computing principles We have p processors available to execute a task T Ideally: the more processors the faster a task is executed Reality: synchronisation and communication costs Speedup s(t, p) of a task T by using p processors: Be t(t, p) the time needed to execute T using p processors Speedup is given by: s(t, p) = t(t, 1) t(t, p) Map Reduce I 5 / 32

11 3. Parallel programming paradigms Parallel Computing principles We have p processors available to execute a task T Efficiency e(t, p) of a task T by using p processors: e(t, p) = t(t, 1) p t(t, p) Map Reduce I 6 / 32

12 3. Parallel programming paradigms Considerations It is not worth using a lot of processors for solving small problems Algorithms should increase efficiency with problem size Map Reduce I 7 / 32

13 3. Parallel programming paradigms Paradigms - Shared Memory All the processors have access to all the data D := {d 1,..., d n } Pieces of data can be overwritten Processors need to lock datapoints before using them For each processor p: 1. lock(d i ) 2. process(d i ) 3. unlock(d i ) Map Reduce I 8 / 32

14 3. Parallel programming paradigms Word Count Example Given a corpus of text documents D := {d 1,..., d n } each containing a sequence of words: w 1,..., w m pooled from a set W of possible words. the task is to generate word counts for each word in the corpus Map Reduce I 9 / 32

15 3. Parallel programming paradigms Word Count - Shared Memory Shared vector for word counts: c R W c {0} W Each processor: 1. access a document d D 2. for each word w i in document d: 3. lock(c i ) 4. c i c i unlock(c i ) Map Reduce I 10 / 32

16 3. Parallel programming paradigms Paradigms - Shared Memory Inefficient in a distributed scenario Results of a process can easily be overwritten Possible long waiting times for a piece of data because of the lock mechanism Map Reduce I 11 / 32

17 3. Parallel programming paradigms Paradigms - Message passing Each processor sees only one part of the data π(d, p) := {d p,..., d p+ n p 1 } Each processor works on its partition Results are exchanged between processors (message passing) For each processor p: 1. For each d π(d, p) 2. process(d) 3. Communicate results Map Reduce I 12 / 32

18 3. Parallel programming paradigms Word Count - Message passing We need to define two types of processes: 1. Slave - counts the words on a subset of documents and informs the master 2. Master - gathers counts from the slaves and sums them up Map Reduce I 13 / 32

19 3. Parallel programming paradigms Word Count - Message passing Slave: Local memory: 1. subset of documents: π(d, p) := {d p,..., d p+ n p 1 } 2. Address of the master: addr master 3. Local word counts: c R W c {0} W 1. for each document d π(d, p) 2. for each word w i in document d: 3. c i c i + 1 Send message send(addr master, c) Map Reduce I 14 / 32

20 3. Parallel programming paradigms Word Count - Message passing Master: Local memory: 1. Global word counts: c global R W 2. List of slaves: S c global {0} W s {0} S For each received message (p, c p ) 1. c global c global + c p 2. s p 1 3. if s 1 = S return c global Map Reduce I 15 / 32

21 3. Parallel programming paradigms Paradigms - Message passing We need to manually assign master and slave roles for each processor Partition of the data needs to be done manually Implementations like OpenMPI only provide services to exchange messages Map Reduce I 16 / 32

22 Outline 1. Introduction 2. Parallel Computing 3. Parallel programming paradigms Map Reduce I 17 / 32

23 Map-Reduce Builds on the distributed message passing paradigm Considers the data is partitioned over the nodes Pipelined procedure: 1. Map phase 2. Reduce phase High level abstraction: programmer only specifies a map and a reduce routine Map Reduce I 17 / 32

24 Map-Reduce No need to worry about how many processors are available No need to specify which ones will be mappers and which ones will be reducers Map Reduce I 18 / 32

25 Key-Value input data Map-Reduce requires the data to be stored in a key-value format Natural if one works with column databases Examples: Key document document user user user Value array of words word movies friends tweet Map Reduce I 19 / 32

26 The Paradigm - Formally Given A set of input keys I A set of output keys O A set of input values X A set of intermediate values V A set of output values Y We can define: map : I X P(O V ) and reduce : O P(V ) O Y where P denotes the powerset Map Reduce I 20 / 32

27 The Paradigm - Unformal 1. Each mapper transforms a set key-value pairs into a list of output keys and intermediate value pairs 2. all intermediate values are grouped according to their output keys 3. each reducer receives all the intermediate values associated with a given keys 4. each reducer associates one final value to each key Map Reduce I 21 / 32

28 Word Count Example Map: Input: document-word list pairs Output: word-count pairs Reduce: Input: word-(count list) pairs Output: word-count pairs (d k, w 1,..., w m) [(w i, c i )] (w i, [c i ]) (w i, c [c i ] c) Map Reduce I 22 / 32

29 Word Count Example (d1, love ain't no stranger ) (d2, crying in the rain ) (love, 1) (stranger,1) (crying, 1) (rain, 1) (ain't, 1) (stranger,1) (love, 1) (love, 2) (love, 2) (crying, 1) (stranger,1) (love, 5) (crying, 2) (crying, 1) (d3, looking for love ) (love, 2) (d4, I'm crying ) (d5, the deeper the love ) (crying, 1) (looking, 1) (deeper, 1) (ain't, 1) (ain't, 2) (ain't, 1) (rain, 1) (rain, 1) (looking, 1) (d6, is this love ) (love, 2) (ain't, 1) (looking, 1) (deeper, 1) (deeper, 1) (this, 1) (d7, Ain't no love ) (this, 1) (this, 1) Mappers Reducers Map Reduce I 23 / 32

30 Map p u b l i c s t a t i c c l a s s Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, I n t W r i t a b l e > { p r i v a t e f i n a l s t a t i c I n t W r i t a b l e one = new I n t W r i t a b l e ( 1 ) ; p r i v a t e Text word = new Text ( ) ; p u b l i c v o i d map( L o n g W r i t a b l e key, Text v a l u e, O u t p u t C o l l e c t o r <Text, I n t W r i t a b l e > output, R e p o r t e r r e p o r t e r ) throws I O E x c e p t i o n { S t r i n g l i n e = v a l u e. t o S t r i n g ( ) ; S t r i n g T o k e n i z e r t o k e n i z e r = new S t r i n g T o k e n i z e r ( l i n e ) ; } } w h i l e ( t o k e n i z e r. hasmoretokens ( ) ) { word. s e t ( t o k e n i z e r. nexttoken ( ) ) ; o u t p u t. c o l l e c t ( word, one ) ; } Map Reduce I 24 / 32

31 Reduce p u b l i c s t a t i c c l a s s Reduce extends MapReduceBase implements Reducer<Text, I n t W r i t a b l e, Text, I n t W r i t a b l e > { p u b l i c v o i d r e d u c e ( Text key, I t e r a t o r <I n t W r i t a b l e > v a l u e s, O u t p u t C o l l e c t o r <Text, I n t W r i t a b l e > output, R e p o r t e r r e p o r t e r ) throws I O E x c e p t i o n { } } i n t sum = 0 ; w h i l e ( v a l u e s. hasnext ( ) ) { sum += v a l u e s. n e x t ( ). g e t ( ) ; } o u t p u t. c o l l e c t ( key, new I n t W r i t a b l e ( sum ) ) ; Map Reduce I 25 / 32

32 Execution snippet p u b l i c s t a t i c v o i d main ( S t r i n g [ ] a r g s ) throws E x c e p t i o n { JobConf c o n f = new JobConf ( WordCount. c l a s s ) ; c o n f. setjobname ( wordcount ) ; c o n f. s e t O u t p u t K e y C l a s s ( Text. c l a s s ) ; c o n f. s e t O u t p u t V a l u e C l a s s ( I n t W r i t a b l e. c l a s s ) ; c o n f. s e t M a p p e r C l a s s (Map. c l a s s ) ; c o n f. s e t C o m b i n e r C l a s s ( Reduce. c l a s s ) ; c o n f. s e t R e d u c e r C l a s s ( Reduce. c l a s s ) ; c o n f. s e t I n p u t F o r m a t ( TextInputFormat. c l a s s ) ; c o n f. setoutputformat ( TextOutputFormat. c l a s s ) ; F i l e I n p u t F o r m a t. s e t I n p u t P a t h s ( conf, new Path ( a r g s [ 0 ] ) ) ; F i l e O u t p u t F o r m a t. setoutputpath ( conf, new Path ( a r g s [ 1 ] ) ) ; } } J o b C l i e n t. runjob ( c o n f ) ; Map Reduce I 26 / 32

33 Considerations Maps are executed in parallel Reduces are executed in parallel Bottleneck: Reducers can only execute after all the mappers are finished Map Reduce I 27 / 32

34 Fault tolerance When the master node detecs node failures: Re-executes completed and in-progress map() Re-executes in-progress reduce tasks When the master node detects particular key-value pairs that causes mappers to crash: Problematic pairs are skipped in the execution Map Reduce I 28 / 32

35 Parallel Efficiency of Map-Reduce We have p processors for performing map and reduce operations Time to perform a task T on data D: t(t, 1) = wd Time for producing intermediate data σd after the map phase: t(t inter, 1) = σd Overheads: intermediate data per mapper: σd p each of the p reducers needs to read one p-th of the data written by each of the p mappers: σd p 1 p p = σd p Time for performing the task with Map-reduce: t MR (T, p) = wd p + 2K σd p K - constant for representing the overhead of IO operations (reading and writing data to disk) Map Reduce I 29 / 32

36 Parallel Efficiency of Map-Reduce Time for performing the task in one processor: wd Time for performing the task with p processors on Map-reduce: t MR (T, p) = wd p + K σd p Efficiency equation: Efficiency of Map-Reduce: e(t, p) = t(t, 1) p t(t, p) e MR (T, p) = wd p( wd p + 2K σd p ) Map Reduce I 30 / 32

37 Parallel Efficiency of Map-Reduce e MR (T, p) = wd p( wd p + 2K σd p ) = wd wd + 2KσD = wd 1 wd wd 1 wd + 2KσD 1 wd 1 = 1 + 2K σ w Map Reduce I 31 / 32

38 Parallel Efficiency of Map-Reduce 1 e MR (T, p) = 1 + 2K σ w Apparently the efficiency is independent of p Large efficiency can be achieved with large number of processors If σ is high (too much intermediate data) the efficiency suffers In many cases σ depends on p Map Reduce I 32 / 32