Lab Course: distributed data analytics

Transcription

1 Lab Course: distributed data analytics 01. Threading and Parallelism Nghia Duong-Trung, Mohsan Jameel Information Systems and Machine Learning Lab (ISMLL) University of Hildesheim, Germany International Master s Program in Data Analytics Summer Semester 2017 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 1 / 49

2 Outline 1. Getting Started with Threading and Parallel Computing Hands-On Multiprocessing Programming Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 2 / 49

3 Getting Started with Threading and Parallel Computing Outline 1. Getting Started with Threading and Parallel Computing Hands-On Multiprocessing Programming Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 3 / 49

4 Getting Started with Threading and Parallel Computing Why Parallel Computing is Important? A good tutorial on parallel computing is available at Get results in a short timespan. process large amount of data in fraction of time, run complex algorithm faster, exploit underline hardware more efficiently. Parallel computing on multicore systems. modern processor technology focus more on multi-cores rather than clock speed, to get true performance, we need to write parallel program. What we need to do? Learn to develop parallel programs, First step: write multi-threaded programs for multicore systems. i.e. learn driving mclaren 720S Advance step: write GPU enable code i.e. learn driving F1 cars Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 3 / 49

5 Getting Started with Threading and Parallel Computing Parallel programming Parallel programming can be defined as an approach in which program data creates workers to run specific tasks simultaneously in a multicore environment without the need for concurrency amongst them to access a CPU [1]. Figure: Parallel programming scheme Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 4 / 49

6 Getting Started with Threading and Parallel Computing Threads Basics Process: an instance of a running program such that each process has its exclusive memory managed by the OS may have several threads heavyweight Thread: a running subprogram such that threads share memory allocated for their parent process managed by the program which creates the threads comprise a complete computing task that cannot be further sub-divided lightweight Multiprocessing: execute several processes in parallel Multithreading: execute several threads in parallel Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 5 / 49

7 Getting Started with Threading and Parallel Computing Threads Basics Imagine the task to multiply a scalar with a matrix In a sequential system, one would perform each multiplication operation one after one and generate the final results at the end of all the instructions Apply the idea of parallelism for the same task: Figure: An example of parallelism in matrix multiplication Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 6 / 49

8 Getting Started with Threading and Parallel Computing States of a thread There are five possible states in a thread s life span: Creation: This is the main process that creates a thread, and after its creation, it is sent to a line of threads ready for execution. Ready: the thread is in a line of threads ready for execution and bounded to be executed. Execution: the thread makes use of the CPU. Blocked: the thread is blocked to wait for an I/O operation and it does not make use of the CPU. Concluded: the thread s life span is end and resources are free from its execution. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 7 / 49

9 Getting Started with Threading and Parallel Computing Parallel Programming Problems Deadlock Starvation A situation in which two or more workers keep indefinitely waiting for the freeing of a required resource which is currently blocked by another worker of the same group. A situation in which one or more processes run heavy tasks and has data processor priority while the others never get the chance. Race Conditions A situation in which the computing result depends on a sequence of operations and this sequence is broken due to the lack of synchronization mechanism. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 8 / 49

10 Getting Started with Threading and Parallel Computing Mechanism of Synchronization An example which contains a mechanism of synchronization Customer 1 Customer 2 A product (units) 10 How many left? 10 Buys 5 10 Concludes operation 5 How many left? 5 Buys 3 5 Concludes operation 2 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 9 / 49

11 Outline 1. Getting Started with Threading and Parallel Computing Hands-On Multiprocessing Programming Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 10 / 49

12 threading module Python manages a thread via the threading module that is provided by the Python standard library. The Python module threading has the Thread() method that is used to run functions in different threads: class threading.thread(group=none, target=none, name=none, args=(), kwargs={}) In the preceding constructor, arguments are: group should be None; this is reserved for future implementations. target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called. name is the thread name. By default, a unique name is constructed of the form Thread-N where N is a small decimal number. args is the argument tuple for the target invocation. Defaults to (). kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 10 / 49

13 threading module To import the threading module, we simply use the Python command: import threading In the main program, we instantiate a thread, pass an argument to the function: t = threading.thread(target=function, args=(i,)) The thread does not start running until the start() method is called, and the join() method makes the calling thread wait until the thread has finished the execution. t.start() t.join() Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 11 / 49

14 threading module 1 import t h r e a d i n g 2 3 d e f f u n c t i o n ( i ) : 4 p r i n t ( f u n c t i o n c a l l e d by t h r e a d {}. format ( i ) ) 5 r e t u r n 6 7 t h r e a d s = [ ] 8 f o r i i n range ( 1 0 ) : 9 t = t h r e a d i n g. Thread ( t a r g e t=f u n c t i o n, a r g s =( i, ) ) 0 t h r e a d s. append ( t ) 1 t. s t a r t ( ) 2 t. j o i n ( ) 3 Listing 1: threading example Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 12 / 49

15 currentthread function 1 i m p o r t t h r e a d i n g 2 i m p o r t time 3 4 d e f f i r s t f u n c t i o n ( ) : 5 p r i n t ( t h r e a d i n g. c u r r e n t T h r e a d ( ). getname ( ) + i s s t a r t i n g \n ) 6 time. s l e e p ( 2 ) #s e c o n d s 7 p r i n t ( t h r e a d i n g. c u r r e n t T h r e a d ( ). getname ( ) + i s e x i t i n g \n ) 8 9 def s e c o n d f u n c t i o n ( ) : 0 p r i n t ( t h r e a d i n g. c u r r e n t T h r e a d ( ). getname ( ) + i s s t a r t i n g \n ) 1 time. s l e e p ( 4 ) #s e c o n d s 2 p r i n t ( t h r e a d i n g. c u r r e n t T h r e a d ( ). getname ( ) + i s e x i t i n g \n ) 3 4 d e f t h i r d f u n c t i o n ( ) : 5 p r i n t ( t h r e a d i n g. c u r r e n t T h r e a d ( ). getname ( ) + i s s t a r t i n g \n ) 6 time. s l e e p ( 2 ) #s e c o n d s 7 p r i n t ( t h r e a d i n g. c u r r e n t T h r e a d ( ). getname ( ) + i s e x i t i n g \n ) 8 9 i f name == main : 0 t1 = t h r e a d i n g. Thread ( name= f i r s t f u n c t i o n, t a r g e t=f i r s t f u n c t i o n ) 1 t2 = t h r e a d i n g. Thread ( name= s e c o n d f u n c t i o n, t a r g e t=s e c o n d f u n c t i o n ) 2 t3 = t h r e a d i n g. Thread ( name= t h i r d f u n c t i o n, t a r g e t=t h i r d f u n c t i o n ) 3 t1. s t a r t ( ) 4 t2. s t a r t ( ) 5 t3. s t a r t ( ) 6 t1. j o i n ( ) 7 t2. j o i n ( ) 8 t3. j o i n ( ) 9 Listing 2: currentthread function Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 13 / 49

16 Why join()? 1 w i t h o u t j o i n : main t h r e a d c h i l d t h r e a d ( s h o r t ) c h i l d t h r e a d ( l o n g ) 6 7 w i t h j o i n ### main t h r e a d j o i n ( ) c h i l d t h r e a d ( s h o r t ) j o i n ( ) c h i l d t h r e a d ( l o n g ) 2 3 with j o i n and demon thread ### parent t h r e a d j o i n ( ) c h i l d t h r e a d ( s h o r t ) j o i n ( ) c h i l d t h r e a d ( l o n g ) 8 +,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, c h i l d t h r e a d ( l o n g+demonized ) 9 0 main thread / parent thread /main program execution 1. c h i l d t h r e a d e x e c u t i o n 2 # o p t i o n a l p a r e nt t h r e a d e x e c u t i o n a f t e r j o i n ( ) b l o c k e d parent t h r e a d c o u l d c o n t i n u e 3 main t h r e a d s l e e p i n g i n j o i n method, w a i t i n g f o r c h i l d t h r e a d to f i n i s h 4, demonized t h r e a d i g n o r e s l i f e t i m e o f o t h e r t h r e a d s ; t e r m i n a t e s when main programs e x i t s ; i s normally meant f o r join independent tasks 5 Listing 3: A simple explanation from stackoverflow Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 14 / 49

17 Inheritance of Thread class To implement a new thread that inherits the Thread class, one has to do: Define a new class that inherits the Thread class. Override the init (self [,args]) and run(self [,args]) to redefine some properties and instructions of the thread if necessary. Once one have created the new Thread subclass, one can create an instance of it and then start by invoking the start() method, which will in turn call the run() method. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 15 / 49

18 Inheritance of Thread class - continued 1 import threading, time 2 3 c l a s s mythread ( t h r e a d i n g. Thread ) : 4 d e f i n i t ( s e l f, thread name, c o u n t e r ) : 5 s e l f. thread name = thread name 6 s e l f. c o u n t e r = c o u n t e r 7 t h r e a d i n g. Thread. i n i t ( s e l f ) 8 9 d e f run ( s e l f ) : 0 d i s p l a y t i m e ( s e l f. name, s e l f. c o u n t e r ) 1 2 d e f d i s p l a y t i m e ( thread name, c o u n t e r ) : 3 while counter : 4 time. s l e e p ( c o u n t e r ) 5 p r i n t ( {}: {}. format ( thread name, time. c t i m e ( time. time ( ) ) ) ) 6 c o u n t e r = thread1 = mythread ( thread 1,5) 0 thread2 = mythread ( thread 2,3) 1 2 t h r e a d 1. s t a r t ( ) 3 t h r e a d 2. s t a r t ( ) 4 t h r e a d 1. j o i n ( ) 5 t h r e a d 2. j o i n ( ) 6 Listing 4: Inheritance of Thread class Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 16 / 49

19 Thread synchronization Determinacy race 1 c o u n t e r = 0 # g l o b a l l y d e f i n e d 2 def process item ( item ) : 3 g l o b a l c o u n t e r 4... do something w i t h item... 5 c o u n t e r += 1 6 Definition: A determinacy race occurs when two logically parallel instructions access the same memory location and at least one of the instructions performs a write Types of Races: Read or Write Tutorial on Python threads synchronization: python-threads-synchronization-locks-rlocks-semaphores-con Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 17 / 49

20 Thread synchronization Atomic operations/ Thread-Safe Operations Lock and RLock Semaphore Condition Event with statement Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 18 / 49

21 Atomic Operations reading or replacing a single instance attribute reading or replacing a single global variable fetching an item from a list modifying a list in place (e.g. adding an item using append) fetching an item from a dictionary modifying a dictionary in place (e.g. adding an item, or calling the clear method) Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 19 / 49

22 Lock When a thread wants to access a portion of shared memory, it must necessarily acquire a lock on that portion prior to using it. After completing its operation, the thread must release the lock previously obtained. The portion of shared memory is available for any other threads. This approach may lead to a bad situation of deadlock. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 20 / 49

23 Without Lock 1 import threading, logging, time 2 l o g g i n g. b a s i c C o n f i g ( l e v e l=l o g g i n g. INFO ) 3 4 s h a r e d r e s o u r c e w i t h n o l o c k, COUNT = 0, def i n c r e m e n t w i t h o u t l o c k ( ) : 7 g l o b a l s h a r e d r e s o u r c e w i t h n o l o c k 8 f o r i i n r a n g e (COUNT) : 9 time. s l e e p ( 2 ) 0 s h a r e d r e s o u r c e w i t h n o l o c k += 1 1 l o g g i n g. i n f o ( r e s o u r c e i s i n c r e a s i n g... + s t r ( s h a r e d r e s o u r c e w i t h n o l o c k ) ) 2 3 def d e c r e m e n t w i t h o u t l o c k ( ) : 4 g l o b a l s h a r e d r e s o u r c e w i t h n o l o c k 5 f o r i i n r a n g e (COUNT) : 6 time. s l e e p ( 2 ) 7 s h a r e d r e s o u r c e w i t h n o l o c k = 1 8 l o g g i n g. i n f o ( r e s o u r c e i s d e c r e a s i n g... + s t r ( s h a r e d r e s o u r c e w i t h n o l o c k ) ) 9 0 i f name == main : 1 t1 = t h r e a d i n g. Thread ( t a r g e t = i n c r e m e n t w i t h o u t l o c k ) 2 t2 = t h r e a d i n g. Thread ( t a r g e t = d e c r e m e n t w i t h o u t l o c k ) 3 4 t1. s t a r t ( ) 5 t2. s t a r t ( ) 6 t1. j o i n ( ) 7 t2. j o i n ( ) 8 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 21 / 49

24 Lock - continued 1 import threading, logging, time 2 l o g g i n g. b a s i c C o n f i g ( l e v e l=l o g g i n g. INFO ) 3 s h a r e d r e s o u r c e w i t h l o c k, COUNT = 0, 5 4 l o c k = t h r e a d i n g. Lock ( ) 5 6 def i n c r e m e n t w i t h l o c k ( ) : 7 g l o b a l s h a r e d r e s o u r c e w i t h l o c k 8 f o r i i n r a n g e (COUNT,0, 1) : 9 l o c k. a c q u i r e ( ) 0 time. s l e e p ( 2 ) 1 s h a r e d r e s o u r c e w i t h l o c k += 1 2 l o g g i n g. i n f o ( r e s o u r c e i s i n c r e a s i n g... + s t r ( s h a r e d r e s o u r c e w i t h l o c k ) ) 3 l o c k. r e l e a s e ( ) 4 5 def d e c r e m e n t w i t h l o c k ( ) : 6 g l o b a l s h a r e d r e s o u r c e w i t h l o c k 7 f o r i i n r a n g e (COUNT,0, 1) : 8 l o c k. a c q u i r e ( ) 9 time. s l e e p ( 2 ) 0 s h a r e d r e s o u r c e w i t h l o c k = 1 1 l o g g i n g. i n f o ( r e s o u r c e i s d e c r e a s i n g... + s t r ( s h a r e d r e s o u r c e w i t h l o c k ) ) 2 l o c k. r e l e a s e ( ) 3 4 i f name == main : 5 t1 = t h r e a d i n g. Thread ( t a r g e t = i n c r e m e n t w i t h l o c k ) 6 t2 = t h r e a d i n g. Thread ( t a r g e t = d e c r e m e n t w i t h l o c k ) 7 t1. s t a r t ( ) 8 t2. s t a r t ( ) 9 t1. j o i n ( ) 0 t2. j o i n ( ) 1 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 22 / 49

25 RLock 1 import threading, time 2 3 c l a s s Box ( o b j e c t ) : 4 d e f i n i t ( s e l f ) : 5 s e l f. t o t a l i t e m s = d e f e x e c u t e ( s e l f, n ) : 8 l o c k. a c q u i r e ( ) 9 s e l f. t o t a l i t e m s += n 0 l o c k. r e l e a s e ( ) 1 2 d e f add ( s e l f ) : 3 l o c k. a c q u i r e ( ) 4 s e l f. e x e c u t e ( 1 ) 5 l o c k. r e l e a s e ( ) 6 7 d e f remove ( s e l f ) : 8 l o c k. a c q u i r e ( ) 9 s e l f. e x e c u t e ( 1) 0 l o c k. r e l e a s e ( ) 1 2 d e f adder ( box, i t e m s ) : 3 w h i l e i t e m s > 0 : 4 l o c k. a c q u i r e ( ) 5 box. add ( ) 6 l o c k. r e l e a s e ( ) 7 p r i n t ( 1 item added to t h e box... \ t \ t + s t r ( box. t o t a l i t e m s ) ) 8 time. s l e e p ( 2 ) 9 i t e m s = 1 0 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 23 / 49

26 RLock - continued 1 d e f remover ( box, i t e m s ) : 2 w h i l e i t e m s > 0 and box. t o t a l i t e m s > 0 : 3 l o c k. a c q u i r e ( ) 4 box. remove ( ) 5 l o c k. r e l e a s e ( ) 6 p r i n t ( 1 item removed from t h e box... \ t + s t r ( box. t o t a l i t e m s ) ) 7 time. s l e e p ( 2 ) 8 9 i f name == main : 0 l o c k = t h r e a d i n g. Lock ( ) 1 i t e m s = 5 2 box = Box ( ) 3 t1 = t h r e a d i n g. Thread ( t a r g e t=adder, a r g s =(box, i t e m s ) ) 4 t2 = t h r e a d i n g. Thread ( t a r g e t=remover, a r g s =(box, i t e m s ) ) 5 t1. s t a r t ( ) 6 t2. s t a r t ( ) 7 t1. j o i n ( ) 8 t2. j o i n ( ) 9 Try to replace lock = threading.lock() by lock = threading.rlock() Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 24 / 49

27 Semaphore A semaphore manages an internal counter which is decremented by each acquire() operation and incremented by each release() operation, as explained: Whenever a thread wants to access a resource that is associated with a semaphore, it must invoke the acquire() operation, which decreases the internal counter of the semaphore. If the value is negative, the thread would be suspended. Whenever a thread has finished using the resource, it must release the resource through the release() operation. the internal counter of the semaphore is incremented. There are many cases we may want to allow more than one worker access to a resource while still limiting the overall number of accesses. One might use semaphore in a situation where we need to support concurrent connections/downloads. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 25 / 49

28 Condition A condition identifies a change of state in the application. A thread waits for a specific condition and another thread notifies that this condition has taken place. Once the condition takes place, the thread acquires the lock to get exclusive access to the shared resource. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 26 / 49

29 Condition - continued 1 from threading import Thread, Condition 2 i m p o r t time 3 4 i t e m s = [ ] 5 c o n d i t i o n = Condition ( ) 6 7 c l a s s consumer ( Thread ) : 8 d e f i n i t ( s e l f ) : 9 Thread. i n i t ( s e l f ) 0 1 d e f consume ( s e l f ) : 2 g l o b a l c o n d i t i o n 3 g l o b a l i t e m s 4 5 c o n d i t i o n. a c q u i r e ( ) 6 7 i f l e n ( i t e m s ) == 0 : 8 p r i n t ( Consumer n o t i f y : no item to consume ) 9 c o n d i t i o n. w a i t ( ) 0 1 i t e m s. pop ( ) 2 p r i n t ( Consumer n o t i f y : consumed 1 item ) 3 p r i n t ( Consumer n o t i f y : i t e m s to consume a r e + s t r ( l e n ( i t e m s ) ) ) 4 c o n d i t i o n. n o t i f y ( ) 5 c o n d i t i o n. r e l e a s e ( ) 6 7 d e f run ( s e l f ) : 8 f o r i i n r a n g e ( 0, 1 0 ) : 9 time. s l e e p ( 5 ) 0 s e l f. consume ( ) 1 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 27 / 49

30 Condition - continued 1 c l a s s p r o d u c e r ( Thread ) : 2 d e f i n i t ( s e l f ) : 3 Thread. i n i t ( s e l f ) 4 5 d e f produce ( s e l f ) : 6 g l o b a l c o n d i t i o n 7 g l o b a l i t e m s 8 9 c o n d i t i o n. a c q u i r e ( ) 0 i f l e n ( i t e m s ) == 5 : 1 p r i n t ( P roducer n o t i f y : i t e m s p r o d u c t e d a r e + s t r ( l e n ( i t e m s ) ) ) 2 p r i n t ( P roducer n o t i f y : s t o p t h e p r o d u c t i o n ) 3 c o n d i t i o n. w a i t ( ) 4 5 items. append ( 1 ) 6 p r i n t ( P roducer n o t i f y : t o t a l i t e m s p r o d u c t e d + s t r ( l e n ( i t e m s ) ) ) 7 c o n d i t i o n. n o t i f y ( ) 8 c o n d i t i o n. r e l e a s e ( ) 9 0 d e f run ( s e l f ) : 1 f o r i i n r a n g e ( 0, 1 0 ) : 2 time. s l e e p ( 2 ) 3 s e l f. produce ( ) 4 5 i f name == main : 6 p = producer ( ) 7 c = consumer ( ) 8 p. s t a r t ( ) 9 c. s t a r t ( ) 0 p. j o i n ( ) 1 c. j o i n ( ) 2 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 28 / 49

31 Event One thread signals an event and other threads wait for it. An event object manages an internal flag that can be set to true with the set() method and reset to false with the clear() method. The wait() method blocks until the flag is true. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 29 / 49

32 Event - continued 1 from threading import Thread, Event 2 import random, time 3 4 c l a s s consumer ( Thread ) : 5 d e f i n i t ( s e l f, items, e v e n t ) : 6 Thread. i n i t ( s e l f ) 7 s e l f. i t e m s = i t e m s 8 s e l f. e v e n t = e v e n t 9 0 d e f run ( s e l f ) : 1 s e l f. e v e n t. w a i t ( ) 2 p r i n t ( Consumer n o t i f y : {} popped from l i s t by {}. format ( s e l f. i t e m s. pop ( ), s e l f. name ) ) 3 s e l f. e v e n t. c l e a r ( ) 4 5 c l a s s p r o d u c e r ( Thread ) : 6 d e f i n i t ( s e l f, i n t e g e r s, e v e n t ) : 7 Thread. i n i t ( s e l f ) 8 s e l f. i t e m s = i t e m s 9 s e l f. e v e n t = e v e n t 0 1 d e f run ( s e l f ) : 2 time. s l e e p ( 2 ) 3 g l o b a l item 4 item = random. randint (0, 100) 5 s e l f. i t e m s. append ( item ) 6 p r i n t ( P roducer n o t i f y : item {} appended to l i s t by {}. format ( item, s e l f. name ) ) 7 s e l f. e v e n t. s e t ( ) 8 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 30 / 49

33 Event - continued 1 i f name == main : 2 i t e m s = [ ] 3 e v e n t = Event ( ) 4 t1 = p r o d u c e r ( items, e v e n t ) 5 t2 = consumer ( items, event ) 6 t1. s t a r t ( ) 7 t2. s t a r t ( ) 8 t1. j o i n ( ) 9 t2. j o i n ( ) 0 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 31 / 49

34 with statement with statement is called a context manager. All objects provided by the acquire() and release() methods may be used in a with statement. The following objects can be used within a with statement. Lock RLock Condition Semaphore Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 32 / 49

35 with statement - continued 1 i m p o r t t h r e a d i n g 2 3 d e f t h r e a d i n g w i t h ( s t a t e m e n t ) : 4 with statement : 5 p r i n t ( s t r ( s t a t e m e n t ) + a c q u i r e d v i a w i t h ) 6 7 def thre ading not with ( statement ) : 8 s t a t e m e n t. a c q u i r e ( ) 9 t r y : 0 p r i n t ( s t r ( s t a t e m e n t ) + a c q u i r e d d i r e c t l y ) 1 f i n a l l y : 2 s t a t e m e n t. r e l e a s e ( ) 3 4 i f name == main : 5 l o c k = t h r e a d i n g. Lock ( ) 6 r l o c k = t h r e a d i n g. RLock ( ) 7 c o n d i t i o n = t h r e a d i n g. Condition ( ) 8 semaphore = threading. Semaphore ( ) 9 t h r e a d i n g s y n c h r o n i z a t i o n l i s t = [ l o c k, r l o c k, c o n d i t i o n, semaphore ] 0 1 f o r s t a t e m e n t i n t h r e a d i n g s y n c h r o n i z a t i o n l i s t : 2 t1 = t h r e a d i n g. Thread ( t a r g e t=t h r e a d i n g w i t h, a r g s =( statement, ) ) 3 t2 = t h r e a d i n g. Thread ( t a r g e t=t h r e a d i n g n o t w i t h, a r g s =( statement, ) ) 4 t1. s t a r t ( ) 5 t2. s t a r t ( ) 6 t1. j o i n ( ) 7 t2. j o i n ( ) 8 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 33 / 49

36 Thread communication using a queue We implement a version of Producer and Consumer with queue. The Producer thread is responsible for putting items into the queue. Consumer thread consumes items if there are any in the queue. We simply consider these queue methods: put(): this puts an item into the queue. get(): this removes an returns an item from the queue. task done(): this needs to be called each time an item has been processed. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 34 / 49

37 Thread communication using a queue - continued 1 from t h r e a d i n g i m p o r t Thread 2 from queue i m p o r t Queue 3 import time, random 4 5 c l a s s p r o d u c e r ( Thread ) : 6 d e f i n i t ( s e l f, queue ) : 7 Thread. i n i t ( s e l f ) 8 s e l f. queue = queue 9 s e l f. name= P roducer 0 1 d e f run ( s e l f ) : 2 f o r i i n r a n g e ( 1 0 ) : 3 item = random. randint (0, 100) 4 s e l f. queue. put ( item ) 5 p r i n t ( P roducer n o t i f y : item {} appended to queue by {}. format ( item, s e l f. name ) ) 6 time. s l e e p ( 1 ) 7 8 c l a s s consumer ( Thread ) : 9 d e f i n i t ( s e l f, queue ) : 0 Thread. i n i t ( s e l f ) 1 s e l f. queue = queue 2 s e l f. name= Consumer 3 4 d e f run ( s e l f ) : 5 w h i l e True : 6 item = s e l f. queue. g e t ( ) 7 p r i n t ( Consumer n o t i f y : {} popped from queue by {}. format ( item, s e l f. name ) ) 8 s e l f. queue. t a s k d o n e ( ) 9 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 35 / 49

38 Thread communication using a queue - continued 1 i f name == main : 2 queue = Queue ( ) 3 t1 = producer ( queue ) 4 t2 = consumer ( queue ) 5 t3 = consumer ( queue ) 6 t4 = consumer ( queue ) 7 t1. s t a r t ( ) 8 t2. s t a r t ( ) 9 t3. s t a r t ( ) 0 t4. s t a r t ( ) 1 t1. j o i n ( ) 2 t2. j o i n ( ) 3 t3. j o i n ( ) 4 t4. j o i n ( ) 5 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 36 / 49

39 Hands-On Multiprocessing Programming Outline 1. Getting Started with Threading and Parallel Computing Hands-On Multiprocessing Programming Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 37 / 49

40 Hands-On Multiprocessing Programming Create a process This section will examine the process-based approach using the standard library multiprocessing. multiprocessing implements the shared memory programming paradigm. That is the programming consists of one or more processors that have access to a common memory. 1 i m p o r t m u l t i p r o c e s s i n g 2 3 d e f f u n c t i o n ( i ) : 4 p r i n t ( c a l l e d f u n c t i o n i n p r o c e s s : {}. format ( i ) ) 5 6 i f name == main : 7 P r o c e s s j o b s = [ ] 8 f o r i i n r a n g e ( 5 ) : 9 p = m u l t i p r o c e s s i n g. P r o c e s s ( t a r g e t=f u n c t i o n, a r g s =( i, ) ) 10 Process jobs. append ( p ) 11 p. s t a r t ( ) 12 p. j o i n ( ) 13 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 37 / 49

41 Hands-On Multiprocessing Programming Name a process It is very useful to associate a name to the processes as debugging an application requires the processes to be well marked and identifiable. 1 import multiprocessing, time 2 3 d e f f o o ( ) : 4 name = m u l t i p r o c e s s i n g. c u r r e n t p r o c e s s ( ). name 5 p r i n t ( S t a r t i n g {}. format ( name ) ) 6 time. s l e e p ( 1 ) 7 p r i n t ( E x i t i n g {}. format ( name ) ) 8 9 i f name == main : 10 p r o c e s s w i t h n a m e = m u l t i p r o c e s s i n g. P r o c e s s ( name= f o o p r o c e s s, t a r g e t=f o o ) 11 p r o c e s s w i t h d e f a u l t n a m e = m u l t i p r o c e s s i n g. P r o c e s s ( t a r g e t=f o o ) 12 p r o c e s s w i t h n a m e. s t a r t ( ) 13 p r o c e s s w i t h d e f a u l t n a m e. s t a r t ( ) 14 p r o c e s s w i t h n a m e. j o i n ( ) 15 p r o c e s s w i t h d e f a u l t n a m e. j o i n ( ) 16 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 38 / 49

42 Hands-On Multiprocessing Programming Kill a process terminate() method kills a process immediately. is alive() method keeps track of whether a process is alive or not. 1 import multiprocessing, time 2 3 d e f f o o ( ) : 4 p r i n t ( S t a r t i n g f u n c t i o n ) 5 time. s l e e p ( 1 ) 6 p r i n t ( F i n i s h e d f u n c t i o n ) 7 8 i f name == main : 9 p = m u l t i p r o c e s s i n g. P r o c e s s ( t a r g e t=f o o ) 0 p r i n t ( P r o c e s s b e f o r e e x e c u t i o n :, p, p. i s a l i v e ( ) ) 1 2 p. s t a r t ( ) 3 p r i n t ( P r o c e s s r u n n i n g :, p, p. i s a l i v e ( ) ) 4 5 p. t e r m i n a t e ( ) 6 p r i n t ( P r o c e s s t e r m i n a t e d :, p, p. i s a l i v e ( ) ) 7 8 p. j o i n ( ) 9 p r i n t ( P r o c e s s j o i n e d :, p, p. i s a l i v e ( ) ) 0 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 39 / 49

43 Hands-On Multiprocessing Programming Inheritance of Process class To implement a new process that inherits the Process class, one has to do: Define a new class that inherits the Process class. Override the init (self [,args]) and run(self [,args]) to redefine some properties and instructions of the process if necessary. Once one have created the new Process subclass, one can create an instance of it and then start by invoking the start() method, which will in turn call the run() method. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 40 / 49

44 Hands-On Multiprocessing Programming Inheritance of Process class - continued 1 import multiprocessing, time 2 3 c l a s s myprocess ( m u l t i p r o c e s s i n g. P r o c e s s ) : 4 d e f i n i t ( s e l f, process name, c o u n t e r ) : 5 s e l f. p r o c e s s n a m e = p r o c e s s n a m e 6 s e l f. c o u n t e r = c o u n t e r 7 m u l t i p r o c e s s i n g. P r o c e s s. i n i t ( s e l f ) 8 9 d e f run ( s e l f ) : 0 d i s p l a y t i m e ( s e l f. name, s e l f. c o u n t e r ) 1 2 d e f d i s p l a y t i m e ( process name, c o u n t e r ) : 3 while counter : 4 time. s l e e p ( c o u n t e r ) 5 p r i n t ( {}: {}. format ( process name, time. c t i m e ( time. time ( ) ) ) ) 6 c o u n t e r = i f name == main : 9 process1 = myprocess ( process 1,5) 0 process2 = myprocess ( process 2,3) 1 2 p r o c e s s 1. s t a r t ( ) 3 p r o c e s s 2. s t a r t ( ) 4 p r o c e s s 1. j o i n ( ) 5 p r o c e s s 2. j o i n ( ) 6 Listing 5: Inheritance of Process class Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 41 / 49

45 Hands-On Multiprocessing Programming Process communication using a queue We implement a version of Producer and Consumer with queue. The Producer process is responsible for putting items into the queue. Consumer process consumes items if there are any in the queue. We simply consider these queue methods: put(): this puts an item into the queue. get(): this removes an returns an item from the queue. task done(): this needs to be called each time an item has been processed. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 42 / 49

46 Hands-On Multiprocessing Programming Process communication using a queue - continued 1 from m u l t i p r o c e s s i n g i m p o r t Process, Queue 2 import time, random 3 4 c l a s s p r o d u c e r ( P r o c e s s ) : 5 d e f i n i t ( s e l f, queue ) : 6 P r o c e s s. i n i t ( s e l f ) 7 s e l f. queue = queue 8 s e l f. name= P roducer 9 0 d e f run ( s e l f ) : 1 f o r i i n r a n g e ( 5 ) : 2 item = random. randint (0, 100) 3 s e l f. queue. put ( item ) 4 p r i n t ( P roducer n o t i f y : item {} appended to queue by {}. format ( item, s e l f. name ) ) 5 p r i n t ( The s i z e o f queue i s {}. format ( s e l f. queue. q s i z e ( ) ) ) 6 time. s l e e p ( 1 ) 7 8 c l a s s consumer ( P r o c e s s ) : 9 d e f i n i t ( s e l f, queue ) : 0 P r o c e s s. i n i t ( s e l f ) 1 s e l f. queue = queue 2 s e l f. name= Consumer 3 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 43 / 49

47 Hands-On Multiprocessing Programming Process communication using a queue - continued 1 d e f run ( s e l f ) : 2 w h i l e True : 3 i f ( s e l f. queue. empty ( ) ) : 4 p r i n t ( The queue i s empty ) 5 b r e a k 6 e l s e : 7 time. s l e e p ( 1 ) 8 item = s e l f. queue. g e t ( ) 9 p r i n t ( Consumer n o t i f y : item {} popped from by {}. format ( item, s e l f. name ) ) 0 p r i n t ( The s i z e o f queue i s {}. format ( s e l f. queue. q s i z e ( ) ) ) 1 time. s l e e p ( 1 ) 2 3 i f name == main : 4 queue = Queue ( ) 5 t1 = producer ( queue ) 6 t2 = consumer ( queue ) 7 t3 = consumer ( queue ) 8 t4 = consumer ( queue ) 9 t1. s t a r t ( ) 0 t2. s t a r t ( ) 1 t3. s t a r t ( ) 2 t4. s t a r t ( ) 3 t1. j o i n ( ) 4 t2. j o i n ( ) 5 t3. j o i n ( ) 6 t4. j o i n ( ) 7 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 44 / 49

48 Hands-On Multiprocessing Programming Process synchronization Timer Lock and RLock Semaphore Condition Event with statement Barrier Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 45 / 49

49 Hands-On Multiprocessing Programming Barrier Barrier divides a program into phases in which it requires all of the processes to reach a barrier before any of the processes proceeds. Code that is executed after a barrier cannot be concurrent with the code executed before the barrier. Figure: Process management with a Barrier. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 46 / 49

50 Hands-On Multiprocessing Programming Barrier - continued 1 import multiprocessing, time, datetime 2 from m u l t i p r o c e s s i n g i m p o r t B a r r i e r, Lock, P r o c e s s 3 4 d e f w i t h b a r r i e r ( s y n c h r o n i z e r, l o c k ) : 5 name = m u l t i p r o c e s s i n g. c u r r e n t p r o c e s s ( ). name 6 s y n c h r o n i z e r. w a i t ( ) 7 now = time. time ( ) 8 with lock : 9 p r i n t ( p r o c e s s {} > {}. format ( name, d a t e t i m e. d a t e t i m e. fromtimestamp ( now ) ) ) 0 1 d e f w i t h o u t b a r r i e r ( ) : 2 name = m u l t i p r o c e s s i n g. c u r r e n t p r o c e s s ( ). name 3 now = time. time ( ) 4 p r i n t ( p r o c e s s {} > {}. format ( name, d a t e t i m e. d a t e t i m e. fromtimestamp ( now ) ) ) 5 6 i f name == main : 7 s y n c h r o n i z e r = B a r r i e r ( 3 ) 8 l o c k = Lock ( ) 9 P r o c e s s ( name= p1 w i t h b a r r i e r, t a r g e t=w i t h b a r r i e r, a r g s =( s y n c h r o n i z e r, l o c k ) ). s t a r t ( ) 0 P r o c e s s ( name= p2 w i t h b a r r i e r, t a r g e t=w i t h b a r r i e r, a r g s =( s y n c h r o n i z e r, l o c k ) ). s t a r t ( ) 1 P r o c e s s ( name= p3 w i t h b a r r i e r, t a r g e t=w i t h b a r r i e r, a r g s =( s y n c h r o n i z e r, l o c k ) ). s t a r t ( ) 2 P r o c e s s ( name= p4 w i t h o u t b a r r i e r, t a r g e t=w i t h o u t b a r r i e r ). s t a r t ( ) 3 P r o c e s s ( name= p5 w i t h o u t b a r r i e r, t a r g e t=w i t h o u t b a r r i e r ). s t a r t ( ) 4 P r o c e s s ( name= p6 w i t h o u t b a r r i e r, t a r g e t=w i t h o u t b a r r i e r ). s t a r t ( ) 5 Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 47 / 49

51 Hands-On Multiprocessing Programming Parallel processes with Pool class The multiprocessing library provides the Pool class for simple parallel processing tasks. 1 i m p o r t m u l t i p r o c e s s i n g 2 3 d e f f u n c t i o n s q u a r e ( data ) : 4 r e t u r n data data 5 6 i f name == main : 7 i n p u t s = l i s t ( r a n g e ( 0, ) ) 8 p o o l = m u l t i p r o c e s s i n g. Pool ( p r o c e s s e s =4) 9 p o o l o u t p u t s = pool. map( f u n c t i o n s q u a r e, i n p u t s ) 10 p o o l. c l o s e ( ) 11 p o o l. j o i n ( ) 12 p r i n t ( p o o l o u t p u t s ) 13 It is important to note that the result of the pool.map() method is equivalent to Python s built-in function map(), except that the processes run parallelly. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 48 / 49

52 Hands-On Multiprocessing Programming Further Reading 1. Palach, Jan. Parallel Programming with Python. Packt Publishing Ltd, Threads in Python 3. The Python threading module 4. The Python multi-processing module 5. The parallel Python module 6. Celery: Distributed Task Queue Lutz, M. (2013). Learning python. O Reilly Media, Inc.. 9. Beazley, D., & Jones, B. K. (2013). Python cookbook. O Reilly Media, Inc Zaccone, G. (2015). Python Parallel Programming Cookbook. Packt Publishing Ltd. 11. Lanaro, G. (2013). Python High Performance Programming. Packt Publishing Ltd. Nghia Duong-Trung, Mohsan Jameel, ISMLL, University of Hildesheim, Germany 49 / 49