INF4140 - Models of concurrency RPC and Rendezvous INF4140 Lecture 15. Nov. 2017
RPC and Rendezvous
Outline More on asynchronous message passing interacting processes with different patterns of communication summary remote procedure calls concept, syntax, and meaning examples: time server, merge filters, exchanging values rendez-vous concept, syntax, and meaning examples: buffer, time server, exchanging values combinations of RPC, rendezvous and message passing Examples: bounded buffer, readers/writers 3 / 1
Interacting peers (processes): exchanging values example Look at processes as peers. Example: Exchanging values Consider n processes P[0],..., P[n 1], n > 1 every process has a number, stored in local variable v Goal: all processes knows the largest and smallest number. simplistic problem, but characteristic of distributed computation and information distribution 4 / 1
Different communication patterns P5 P4 P5 P4 P5 P4 P0 P0 P3 P0 P3 P1 P3 P2 P1 P2 P1 P2 centralized symetrical ring shaped 5 / 1
Centralized solution Process P[0] is the coordinator process: P[0] does the calculation The other processes sends their values to P[0] and waits for a reply. P1 P5 P0 P2 P4 P3 Number of messages: 1 (number of sends:) P[0]: n 1 P[1],..., P[n 1]: (n 1) Total: (n 1) + (n 1) = 2(n 1) 2n messages repeated computation Number of channels: n 1 For now in the pics: 1 line = 1 message (not 1 channel), but the notation in the pics is not 100% consistent. 6 / 1
Centralized solution: code chan v a l u e s ( i n t ), r e s u l t s [ 1.. n 1]( i n t s m a l l e s t, i n t l a r g e s t ) ; p r o c e s s P [ 0 ] { # c o o r d i n a t o r p r o c e s s i n t v :=... ; i n t new, s m a l l e s t := v, l a r g e s t := v ; # i n i t i a l i z a t i o n # g e t v a l u e s and s t o r e t h e l a r g e s t and s m a l l e s t f o r [ i = 1 to n 1] { r e c e i v e v a l u e s ( new ) ; i f ( new < s m a l l e s t ) s m a l l e s t := new ; i f ( new > l a r g e s t ) l a r g e s t := new ; # send r e s u l t s f o r [ i = 1 to n 1] send r e s u l t s [ i ] ( s m a l l e s t, l a r g e s t ) ; p r o c e s s P[ i = 1 to n 1] { i n t v :=... ; i n t s m a l l e s t, l a r g e s t ; send v a l u e s ( v ) ; r e c e i v e r e s u l t s [ i ] ( s m a l l e s t, l a r g e s t ) ; # F i g. 7. 1 1 i n Andrews ( c o r r e c t e d a bug ) 7 / 1
Asymmetric solution in Go (1) f o r i :=0; i <m; i++ { go P ( i, v a l u e s, r e s u l t s [ i ], r ) f o r i :=0; i <m; i++ { v = < v a l u e s i f v > l a r g e s t { l a r g e s t = v fmt. P r i n t f ( " l a r g e s t %v\n", l a r g e s t ) f o r i := range r e s u l t s { r e s u l t s [ i ] < l a r g e s t 8 / 1
Asymmetric solution in Go (2) package main import ( " fmt " ; "math/ rand " ) func P( i i n t, vc chan i n t, r c chan i n t, r rand. Rand ) { v := r. I n t n (100) fmt. P r i n t f ( "P%v\n", v ) vc < v ; fmt. P r i n t f ( "", < r c ) 9 / 1
Symmetric solution P5 P4 P0 P3 P1 P2 Single-programme, multiple data (SPMD) -solution: Each process executes the same code and shares the results with all other processes. Number of messages: n processes sending n 1 messages each, Total: n(n 1) messages. Number of (bi-directional) channels: n(n 1) 10 / 1
Symmetric solution: code chan v a l u e s [ n ] ( i n t ) ; p r o c e s s P[ i = 0 to n 1] { i n t v :=... ; i n t new, s m a l l e s t := v, l a r g e s t := v ; # send v to a l l n 1 o t h e r p r o c e s s e s f o r [ j = 0 to n 1 s t j i ] send v a l u e s [ j ] ( v ) ; # g e t n 1 v a l u e s # and s t o r e t h e s m a l l e s t and l a r g e s t. f o r [ j = 1 to n 1] { # j not used i n t h e l o o p r e c e i v e v a l u e s [ i ] ( new ) ; i f ( new < s m a l l e s t ) s m a l l e s t := new ; i f ( new > l a r g e s t ) l a r g e s t := new ; # F i g. 7. 1 2 from Andrews 11 / 1
Ring solution P5 P4 P0 P3 P1 P2 Almost symmetrical, except P[0], P[n 2] and P[n 1]. Each process executes the same code and sends the results to the next process (if necessary). Number of messages: P[0]: 2 P[1],..., P[n 3]: (n 3) 2 P[n 2]: 1 P[n 1]: 1 2 + 2(n 3) + 1 + 1 = 2(n 1) messages sent. Number of channels: n. 12 / 1
Ring solution: code (1) chan v a l u e s [ n ] ( i n t s m a l l e s t, i n t l a r g e s t ) ; p r o c e s s P [ 0 ] { # s t a r t s t h e exchange i n t v :=... ; i n t s m a l l e s t := v, l a r g e s t := v ; # send v to t h e n e x t p r o c e s s, P [ 1 ] send v a l u e s [ 1 ] ( s m a l l e s t, l a r g e s t ) ; # g e t t h e g l o b a l s m a l l e s t and l a r g e s t from P [ n 1] # and send them to P [ 1 ] r e c e i v e v a l u e s [ 0 ] ( s m a l l e s t, l a r g e s t ) ; send v a l u e s [ 1 ] ( s m a l l e s t, l a r g e s t ) ; 13 / 1
Ring solution: code (2) p r o c e s s P[ i = 1 to n 1] { i n t v :=... ; i n t s m a l l e s t, l a r g e s t ; # g e t s m a l l e s t and l a r g e s t so f a r, # and update them by comparing them to v r e c e i v e v a l u e s [ i ] ( s m a l l e s t, l a r g e s t ) i f ( v < s m a l l e s t ) s m a l l e s t := v ; i f ( v > l a r g e s t ) l a r g e s t := v ; # f o r w a r d t h e r e s u l t, and w a i t f o r t h e g l o b a l r e s u l t send v a l u e s [ ( i +1) mod n ] ( s m a l l e s t, l a r g e s t ) ; i f ( i < n 1) r e c e i v e v a l u e s [ i ] ( s m a l l e s t, l a r g e s t ) ; # f o r w a r d t h e g l o b a l r e s u l t, but not from P [ n 1] to P [ 0 ] i f ( i < n 2) send v a l u e s [ i +1]( s m a l l e s t, l a r g e s t ) ; # F i g. 7. 1 3 from Andrews ( m o d i f i e d ) 14 / 1
Message passing: Summary Message passing: well suited to programming filters and interacting peers (where processes communicates one way by one or more channels). May be used for client/server applications, but: Each client must have its own reply channel In general: two way communication needs two channels many channels RPC and rendezvous are better suited for client/server applications. 15 / 1
Remote Procedure Call: main idea CALLER CALLEE a t computer A a t computer B op foo(formals); # declaration... call foo(args); -----> proc foo(formals) # new process... <----- end;... 16 / 1
RPC (cont.) RPC: combines elements from monitors and message passing As ordinary procedure call, but caller and callee may be on different machines. 2 Caller: blocked until called procedure is done, as with monitor calls and synchronous message passing. Asynchronous programming: not supported directly A new process handles each call. Potentially two way communication: caller sends arguments and receives return values. 2 cf. RMI 17 / 1
RPC: module, procedure, process Module: new program component contains both procedures and processes. module M h e a d e r s o f e x p o r t e d o p e r a t i o n s ; body v a r i a b l e d e c l a r a t i o n s ; i n i t i a l i z a t i o n code ; p r o c e d u r e s f o r e x p o r t e d o p e r a t i o n s ; l o c a l p r o c e d u r e s and p r o c e s s e s ; end M Modules may be executed on different machines M has: procedures and processes may share variables execute concurrently must be synchronized to achieve mutex May only communicate with processes in M by procedures exported by M 18 / 1
RPC: operations Declaration of operation O: op O(formal parameters.) [ returns result] ; Implementation of operation O: proc O(formal identifiers.) [ returns result identifier]{ declaration of local variables; statements Call of operation O in module M: 3 Processes: as before. call M.O(arguments) 3 Cf. static/class methods 19 / 1
Synchronization in modules RPC: primarily a communication mechanism within the module: in principle allowed: more than one process shared data need for synchronization two approaches 1. implicit : as in monitors: mutex built-in additionally condition variables (or semaphores) 2. explicit : 4 user-programmed mutex and synchronization (like semaphorse, local monitors etc.) 4 assumed in the following 20 / 1
Example: Time server (RPC) module providing timing services to processes in other modules. interface: two visible operations: get_time() returns int returns time of day delay(int interval) let the caller sleep a given number of time units multiple clients: may call get_time and delay at the same time Need to protect the variables. internal process that gets interrupts from machine clock and updates tod 21 / 1
Time server code (rpc) module TimeServer op get_time ( ) r e t u r n s i n t ; op d e l a y ( i n t i n t e r v a l ) ; body i n t tod := 0 ; # time o f day sem m := 1 ; # f o r mutex sem d [ n ] := ( [ n ] 0 ) ; # f o r d e l a y e d p r o c e s s e s queue o f ( i n t waketime, i n t p r o c e s s _ i d ) napq ; ## when m = 1, tod < waketime f o r d e l a y e d p r o c e s s e s proc get_time ( ) r e t u r n s time { time := tod ; proc d e l a y ( i n t i n t e r v a l ) { P(m) ; # assume u n i q u e myid and i [ 0, n 1] i n t waketime := tod + i n t e r v a l ; i n s e r t ( waketime, myid ) a t a p p r o p r i a t e p l a c e i n napq ; V(m) ; P( d [ myid ] ) ; # Wait to be awoken p r o c e s s Clock.... end TimeServer 22 / 1
Time server code: clock process p r o c e s s Clock { i n t i d ; s t a r t hardware t i m e r ; w h i l e ( t r u e ) { wait f o r i n t e r r u p t, then r e s t a r t hardware t i m e r tod := tod + 1 ; P(m) ; # mutex w h i l e ( tod s m a l l e s t waketime on napq ) { remove ( waketime, i d ) from napq ; # book k e e p i n g V( d [ i d ] ) ; # awake p r o c e s s V(m) ; end TimeServer # F i g. 8. 1 o f Andrews # mutex 23 / 1
Rendezvous RPC: offers inter-module communication synchronization (often): must be programmed explicitly Rendezvous: known from the language Ada (US DoD) combines communication and synchronization between processes No new process created for each call instead: perform rendezvous with existing process operations are executed one at the time synch_send and receive may be considered as primitive rendezvous. cf. also join-synchronization 24 / 1
Rendezvous: main idea CALLER CALLEE a t computer A a t computer B op foo(formals); # declaration...... # existing process call foo(args); -----> in foo(formals) -> BODY; <----- ni... 25 / 1
Rendezvous: module declaration module M op O 1 ( t y p e s ) ;... op O n ( t y p e s ) ; body p r o c e s s P 1 { v a r i a b l e d e c l a r a t i o n s ; w h i l e ( t r u e ) # s t a n d a r d p a t t e r n i n O 1 ( f o r m a l s ) and B 1 > S 1 ;... [ ] O n ( f o r m a l s ) and B n > S n ; n i... o t h e r p r o c e s s e s end M 26 / 1
Calls and input statements Call: c a l l O i (expr 1,..., expr m ) ; Input statement, multiple guarded expressions: i n O 1 (v 1,... v m1 ) and B 1 > S 1 ;... [ ] O n(v 1,... v mn ) and B n > S n ; n i The guard consists of: and B i synchronization expression (optional) S i statements (one or more) The variables v 1,..., v mi may be referred by B i and S i may read/write to them. 5 5 once again: no side-effects in B!!! 27 / 1
Semantics of input statement Consider the following: i n... [ ] O i (v i,..., v mi ) and B i > S i ;... n i The guard succeeds when O i is called and B i is true (or omitted). Execution of the in statement: Delays until a guard succeeds If more than one guard succeed, the oldest call is served 6 Values are returned to the caller The the call- and in-statements terminates 6 this may be changed using additional syntax (by), see [Andrews, 2000]. 28 / 1
Different variants different versions of rendezvous, depending on the language origin: ADA (accept-statement) (see [Andrews, 2000, Section 8.6]) design variation points synchronization expressions or not? scheduling expressions or not? can the guard inspect the values for input variables or not? non-determinism checking for absence of messages? priority checking in more than one operation? 29 / 1
Bounded buffer with rendezvous module BoundedBuffer op d e p o s i t ( TypeT ), f e t c h ( r e s u l t TypeT ) ; body p r o c e s s B u f f e r { elem buf [ n ] ; i n t f r o n t := 0, r e a r := 0, count := 0 ; w h i l e ( t r u e ) i n d e p o s i t ( item ) and count < n > buf [ r e a r ] := item ; count++; r e a r := ( r e a r +1) mod n ; [ ] f e t c h ( item ) and count > 0 > item := buf [ f r o n t ] ; count ; f r o n t := ( f r o n t +1) mod n ; n i end BoundedBuffer # F i g. 8. 5 o f Andrews 30 / 1
Example: time server (rendezvous) module TimeServer op get_time ( ) r e t u r n s i n t ; op d e l a y ( i n t ) ; # a b s o l u t e waketime as argument op t i c k ( ) ; # c a l l e d by t h e c l o c k i n t e r r u p t h a n d l e r body p r o c e s s Timer { i n t tod := 0 ; s t a r t t i m e r ; w h i l e ( t r u e ) i n get_time ( ) r e t u r n s time > time := tod ; [ ] d e l a y ( waketime ) and waketime <= tod > s k i p ; [ ] t i c k ( ) > { tod++; r e s t a r t t i m e r ; n i end TimeServer # F i g. 8. 7 o f Andrews 31 / 1
RPC, rendezvous and message passing We do now have several combinations: invocation service effect call proc procedure call (RPC) call in rendezvous send proc dynamic process creation, asynchronous proc. send in asynchronous message passing in addition (not in Andrews) asynchronous procedure call, wait-by-necessity, futures 32 / 1
Rendezvous, message passing and semaphores Comparing input statements and receive: in O(a 1,...,a n ) ->v 1 =a 1,...,v n =a n ni receive O(v 1,..., v n ) Comparing message passing and semaphores: send O() and receive O() V(O) and P(O) 33 / 1
Bounded buffer: procedures and semaphores (simulated by channels) module BoundedBuffer op d e p o s i t ( typet ), f e t c h ( r e s u l t typet ) ; body elem buf [ n ] ; i n t f r o n t = 0, r e a r = 0 ; # l o c a l o p e r a t i o n to s i m u l a t e semaphores op empty ( ), f u l l ( ), mutexd ( ), mutexf ( ) ; # o p e r a t i o n s send mutexd ( ) ; send mutexf ( ) ; # i n i t. " semaphores " to 1 f o r [ i = 1 to n ] # i n i t. empty "semaphore " to n send empty ( ) ; proc d e p o s i t ( item ) { r e c e i v e empty ( ) ; r e c e i v e mutexd ( ) ; buf [ r e a r ] = item ; r e a r = ( r e a r +1) mod n ; send mutexd ( ) ; send f u l l ( ) ; proc f e t c h ( item ) { r e c e i v e f u l l ( ) ; r e c e i v e mutexf ( ) ; item = buf [ f r o n t ] ; f r o n t = ( f r o n t +1) mod n ; send mutexf ( ) ; send empty ( ) ; end BoundedBuffer # F i g. 8. 1 2 o f Andrews 34 / 1
Bounded buffer with rendezvous module BoundedBuffer op d e p o s i t ( TypeT ), f e t c h ( r e s u l t TypeT ) ; body p r o c e s s B u f f e r { elem buf [ n ] ; i n t f r o n t := 0, r e a r := 0, count := 0 ; w h i l e ( t r u e ) i n d e p o s i t ( item ) and count < n > buf [ r e a r ] := item ; count++; r e a r := ( r e a r +1) mod n ; [ ] f e t c h ( item ) and count > 0 > item := buf [ f r o n t ] ; count ; f r o n t := ( f r o n t +1) mod n ; n i end BoundedBuffer # F i g. 8. 5 o f Andrews 35 / 1
The primitive?o in rendezvous New primitive on operations, similar to empty(... ) for condition variables and channels.?o means number of pending invocations of operation O. Useful in the input statement to give priority: i n [ ] O 1... > S 1 ; O 2... and (?O 1 = 0) > S 2 ; n i Here O 1 has a higher priority than O 2. 36 / 1
Readers and writers module ReadersWriters op r e a d ( r e s u l t t y p e s ) ; # u s e s RPC op w r i t e ( t y p e s ) ; # u s e s r e n d e z v o u s body op s t a r t r e a d ( ), e n d r e ad ( ) ; # l o c a l ops.... d a t a b a s e (DB )... ; proc r e a d ( v a r s ) { c a l l s t a r t r e a d ( ) ; # g e t r e a d a c c e s s... r e a d v a r s from DB... ; send e n d read ( ) ; # f r e e DB p r o c e s s W r i t e r { i n t nr := 0 ; w h i l e ( t r u e ) end i n s t a r t r e a d ( ) > nr++; [ ] e n dread ( ) > nr ; [ ] w r i t e ( v a r s ) and nr = 0 >... w r i t e v a r s to DB... ; n i ReadersWriters 37 / 1
Readers and writers: prioritize writers module R e a d e r s W r i t e r s op r e a d ( r e s u l t typet ) ; # u s e s RPC op w r i t e ( typet ) ; # u s e s r e n d e z v o u s body op s t a r t r e a d ( ), e n d r e ad ( ) ; # l o c a l ops.... d a t a b a s e (DB )... ; proc r e a d ( v a r s ) { c a l l s t a r t r e a d ( ) ; # g e t r e a d a c c e s s... r e a d v a r s from DB... ; send e n d read ( ) ; # f r e e DB p r o c e s s W r i t e r { i n t nr := 0 ; w h i l e ( t r u e ) i n s t a r t r e a d ( ) and? w r i t e = 0 > nr++; [ ] e n dread ( ) > nr ; [ ] w r i t e ( v a r s ) and nr = 0 >... w r i t e v a r s to DB... ; n i end R e a d e r s W r i t e r s 38 / 1
References I [int, 2013] (2013). Intel 64 and IA-32 Architectures Software Developer s Manual. Combined Volumes:1, 2A, 2B, 2C, 3A, 3B and 3C. Intel. [Adve and Gharachorloo, 1995] Adve, S. V. and Gharachorloo, K. (1995). Shared memory consistency models: A tutorial. Research Report 95/7, Digital WRL. [Adve and Hill, 1990] Adve, S. V. and Hill, M. D. (1990). Weak ordering a new definition. SIGARCH Computer Architecture News, 18(3a). [Anderson, 1990] Anderson, T. E. (1990). The performance of spin lock alternatives for shared-memory multiprocessors. IEEE Transactions on Parallel and Distributed System, 1(1):6 16. [Andrews, 2000] Andrews, G. R. (2000). Foundations of Multithreaded, Parallel, and Distributed Programming. Addison-Wesley. [Go memory model, 2016] Go memory model (2016). The Go memory model. https://golang.org/ref/mem. [Herlihy and Shavit, 2008] Herlihy, M. and Shavit, N. (2008). The Art of Multiprocessor Programming. Morgan Kaufmann. 39 / 1
References II [Lamport, 1979] Lamport, L. (1979). How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Transactions on Computers, C-28(9):690 691. [Manson et al., 2005] Manson, J., Pugh, W., and Adve, S. V. (2005). The Java memory memory. In Proceedings of POPL 05. ACM. [Owell et al., 2009] Owell, S., Sarkar, S., and Sewell, P. (2009). A better x86 memory model: x86-tso. In Berghofer, S., Nipkow, T., Urban, C., and Wenzel, M., editors, Theorem Proving in Higher-Order Logic: 10th International Conference, TPHOLs 09, volume 5674 of Lecture Notes in Computer Science. [Sewell et al., 2010] Sewell, P., Sarkar, S., Nardelli, F., and O.Myreen, M. (2010). x86-tso: A rigorous and usable programmer s model for x86 multiprocessors. Communications of the ACM, 53(7). 40 / 1