Stochastic Automata Networks: Product Forms and Iterative Solutions.

INSTITUT NATIONAL DE REHERHE EN INFORMATIQUE ET EN AUTOMATIQUE Stochastic Automata Networks: Product Forms and Iterative Soutions. rigitte Pateau and Wiiam J. Stewart N 2939 2 juiet 996 THÈME apport de recherche ISSN 249-6399

Stochastic Automata Networks: Product Forms and Iterative Soutions. rigitte Pateau and Wiiam J. Stewart y Theme Reseaux et systemes Projet APAHE Rapport de recherche n2939 2 juiet 996 38 pages Abstract: This artice presents a goba overview of recent resuts concerning stochastic automata networks. Among the topics considered is the formaism of an extended tensor agebra, the rigorous denition of a Markovian generator in the form of a descriptor, suf- cient conditions for product form, the compexity of the vector-descriptor mutipication, the optimization of this product and some numerica resuts. The whoe is iustrated by numerous typica exampes. Key-words: Markov chains, Stochastic automata networks, Product forms, Iterative soutions, Vector-descriptor mutipication. (Resume : tsvp) IMAG{LM, rue des Mathematiques, 384 Grenobe cedex, France. Research supported by the (NRS { INRIA { INPG { UJF) joint project Apache. y Department of omputer Science, N. aroina State University, Raeigh, N.. 27695-826, USA. Research supported in part by NSF (DDM-896248 and R-94339). Unité de recherche INRIA Rhône-Apes 655, avenue de Europe, 3833 MONTONNOT ST MARTIN (France) Tééphone : (33) 76 6 52 Téécopie : (33) 76 6 52 52

Les Reseaux d'automates Stochastiques: Formes Produits et Soutions Iteratives. Resume : et artice presente une vue d'ensembe de resutats recents sur es automates stochastiques: e formaisme de 'agebre tensoriee etendue, a formaisation du generateur Markovien sous forme de descriptor, des conditions susantes de formes produits, a compexite du produit vecteur-descripteur, des optimisations de ce produit et queques resutats numeriques. e travai est iustre par de nombreux exempes typiques. Mots-ce : ha^ne de Markov, Reseau d'automates stochastiques, Formes produits, Produit vecteur-descripteur.

Stochastic Automata Networks: Product Forms and Iterative Soutions. 3 Introduction A Stochastic Automata Network (SAN) consists of a number of individua stochastic automata that operate more or ess independenty of each other. Each individua automaton, A, is represented by a number of states and rues that govern the manner in which it moves from one state to the next. The state of an automaton at any time t is just the state it occupies at time t and the state of the SAN at time t is given by the state of each of its constituent automata. The use of stochastic automata networks is becoming increasingy important in performance modeing issues reated to parae and distributed computer systems. As such modes become increasingy compex, so aso does the compexity of the modeing process. Athough systems anaysts have a number of other modeing strategems at their disposa, it is not unusua to discover that these are inadequate. The use of queueing network modeing is imited by the constraints imposed by assumptions needed to keep the mode tractabe. The resuts obtained from the myriad of avaiabe approximate soutions are frequenty too gross to be meaningfu. Simuations can be excessivey expensive. This eaves modes that are based on Markov chains, but here aso, the dicuties are we documented. The size of the state space generated is so arge that it eectivey prohibits the computation of a soution. This is true whether the Markov chain resuts from a stochastic Petri net formaism, or from a straightforward Markov chain anayzer. In many instances, the SAN formaism is an appropriate choice. Parae and distributed systems are often viewed as coections of components that operate more or ess independenty, requiring ony infrequent interaction such as synchronizing their actions, or operating at dierent rates depending on the state of parts of the overa system. This is exacty the viewpoint adopted by SANs. The components are modeed as individua stochastic automata that interact with each other. Furthermore, the state space exposion probem associated with Markov chain modes is mitigated by the fact that the state transition matrix is not stored, nor even generated. Instead, it is represented by a number of much smaer matrices, one for each of the stochastic automata that constitute the system, and from these a reevent information may be determined without expicity forming the goba matrix. The impication is that a considerabe saving in memory is eected by storing the matrix in this fashion. We do not wish to give the impression that we regard SANs as a panacea for a modeing probems, just that there is a niche that it s among the toos that modeers may use. It is fairy obvious that their memory requirements are minima; it remains to show that this does not come at the cost of a prohibitive amount of computation time. Stochastic Automata Networks and the reated concept of Stochastic Process Agebras have become a hot topic of research in recent years. This research has focused on areas such as the deveopment of anguages for specifying SANs and their ik, [23, 24], and on the deveopment of suitabe soution methods that can operate on the transition matrix given as a compact SAN descriptor. The deveopment of anguages for specifying stochastic process agebras is mainy concerned with structura properties of the nets (compositionaity, equivaence, etc.) and with the mapping of these specications onto Markov chains for the computation of performance measures [24, 3, 7]. Athough a SAN may be viewed as a RR n2939

4 rigitte Pateau and Wiiam J. Stewart stochastic process agebra, its origina purpose was to provide an ecient and convenient methodoogy for the study of quantitive rather than structura properties of compex systems, [32]. Nevertheess, computationa resuts such as those discussed in this artice can aso be appied in the context of stochastic process agebras. There are two overriding concerns in the appication of any Markovian modeing methodoogy, viz., memory requirements and computation time. Since these are frequenty functions of the number of states, a rst approach is to deveop techniques that minimize the number of states in the mode. In SANs, it is possibe to make use of symmetries as we as umping and various superpositioning of the automata to reduce the computationa burden, [, 9, 39]. Furthermore, in [7], structura properties of the Markov chain graph (speciciay the occurrence of cyces) are used to compute steady state soutions. We point out that simiar, and even more extensive resuts have previousy been deveoped in the context of Petri nets and stochastic activity networks. For exampe, in [8, 9,, 8, 38, 4], equivaence reations and symmetries are used to decrease the computationa burden of obtaining performance indices. In [2], reduction techniques for Petri nets are used in conjunction with insensitivity resuts to enabe a computations to be performed on a reduced set of markings. In [], neary independent subnets are expoited in an iterative procedure in which a goba soution is obtained from partia soutions. In [3] it is shown that the tensor structure of the transition matrix may be extracted from a stochastic Petri net, and in [27] that this can be used ecienty to work with the reachabe state space in an iterative procedure. Once the number of states has eectivey been xed, the probem of memory and computation time sti must be addressed, for the number of states eft may sti be arge. With SANs, the use of a compact descriptor goes a ong way to satisfying the rst of these, athough with the need to keep a minimum of two vectors of ength equa to the goba number of states, and consideraby more than two for more sophisticated procedures such as the GMRES method, we cannot aord to become compacent about memory requirements. As far as computation time is concerned, since the numerica methods used are iterative, it is important to keep both the number of iterations and the amount of computation per iteration to a minimum. Our paper is aid out as foows. Sections 2 through 4 present an informa introduction to SANs and tensor agebra. This is foowed in Section 5 with the presentation of a number of sucient conditions for the existence of product forms in SANs, for in some restricted cases, product forms can indeed be found. We note in passing that in [4, 6, 4, 9, 22, 28] product forms have been found in Petri nets modes, using either the structure of the state space or ow properties. The numerica issues of computation time and memory requirements in computing stationary distributions by means of iterative methods are discussed in Sections 6 through. INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 5 2 asic Properties of Tensor Agebra Dene two matrices A and as foows: a a A = 2 a 2 a 22 and = b b 2 b 3 b 4 b 2 b 22 b 23 b 24 b 3 b 32 b 33 b 34 A : The tensor product = A is given by a = a 2 a 2 a 22 () In genera, to dene the tensor product of two matrices, A of dimensions ( ) and of dimensions ( 2 2 ), it is convenient to observe that the tensor product matrix has dimensions ( 2 2 ) and may be considered as consisting of bocks each having dimensions ( 2 2 ), i.e., the dimensions of. To specify a particuar eement, it suces to specify the bock in which the eement occurs and the position within that bock of the eement under consideration. Thus, in the above exampe, the eement c 47 (= a 22 b 3 ) is in the (2; 2) bock and at position (; 3) of that bock. The tensor sum of two square matrices A and is dened in terms of tensor products as A = A I n2 + I n (2) where n is the order of A; n 2 the order of ; I ni the identity matrix of order n i and \+" represents the usua operation of matrix addition. Since both sides of this operation (matrix addition) must have identica dimensions, it foows that tensor addition is dened for square matrices ony. For exampe, with a a A = 2 a 2 a 22 the tensor sum = A is given by = and = b b 2 b 3 b 2 b 22 b 23 b 3 b 32 b 33 a + b b 2 b 3 a 2 b 2 a + b 22 b 23 a 2 b 3 b 32 a + b 33 a 2 a 2 a 22 + b b 2 b 3 a 2 b 2 a 22 + b 22 b 23 a 2 b 3 b 32 a 22 + b 33 Some important properties of tensor products and additions are Associativity: A ( ) = (A ) and A ( ) = (A ). A ; (3) : A RR n2939

6 rigitte Pateau and Wiiam J. Stewart Distributivity over (ordinary matrix) addition: (A + ) ( + D) = A + + A D + D. ompatibiity with (ordinary matrix) mutipication: (A ) ( D) = (A ) ( D). ompatibiity with (ordinary matrix) inversion: (A )? = A??. N L N The associativity property impies that the operations k= A(k) N and k= A(k) are we dened. In particuar, observe that the tensor sum of N terms may be written as the (usua) matrix sum of N terms, each term consisting of an N-fod tensor product. We have NM k= A (k) = NX k= I n I nk? A (k) I nk+ I nn ; where n k is the order of the matrix A (k) and I nk is the identity matrix of order n k. The operators and are not commutative. However, we sha have need of a pseudocommutativity property that may be formaized as foows. Let be a permutation Q of the N set of integer [; 2; : : : ; N]. Then there exists a permutation matrix, P, of order n i= i, such that NO k= A (k) = P N O k= A ((k)) P T : A proof of this property may be found in [33] wherein P is expicity given. Further information concerning the properties of tensor agebra may be found in Davio [2]. 3 Stochastic Automata Networks 3. Non-Interacting Stochastic Automata onsider the case of a system that may be modeed by two competey independent stochastic automata, each of which may be represented by a discrete-time Markov chain. Let us assume that the rst automaton, denoted A (), has n states and stochastic transition probabiity matrix given by P () 2 R nn. Simiary, et A (2) denote the second automaton; n 2, the number of states in its representation and P (2) 2 R n2n2, its stochastic transition probabiity matrix. The state of the overa (two-dimensiona) system may be represented by the pair (i; j) where i 2 f; 2; : : : ; n g and j 2 f; 2; : : : ; n 2 g. Indeed the stochastic transition probabiity matrix of the two-dimensiona system is given by the tensor product of the matrices P () and P (2). If, instead of being represented by two discrete-time Markov chains, the stochastic automata are characterized by continuous-time Markov chains with innitesima generators, Q () and Q (2) respectivey, the innitesima generator of the twodimensiona system is given by the tensor sum of Q () and Q (2). Throughout this artice, INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 7 we present resuts on the basis of continuous-time SANs, athough the resuts are equay vaid in the context of discrete-time SANs. ontinuing with the exampe, et () i (t) be the probabiity that the rst automaton is in state i at time t and (2) j (t) the probabiity that the second is in state j, at time t. Then the probabiity that, at time t, the rst is in state i and the second is in state j is simpy the product () i (t) (2) j (t). Furthermore, the probabiity distribution of the overa (two-dimensiona) system is given by the tensor product of the two individua probabiity vectors, () 2 R n and (2) 2 R n2, viz: () (2). Now given N independent stochastic automata, A () ; A (2) ; : : : ; A (N), with associated innitesima generators, Q () ; Q (2) ; : : : ; Q (N), and probabiity distributions () (t); (2) (t); : : : ; (N) (t) at time t, the innitesima generator of the N{dimensiona system, which we sha refer to as the goba generator, is given by Q = NM k= Q (k) = NX k= I n I nk? Q (k) I nk+ I nn : The probabiity that the system is in state (i ; i 2 ; : : : ; i N ) at time t, where i k is the state of the k th automaton at Q time t with i k n k and n k is the number of states in the k th N automaton, is given by k= (k) i k (t) where (k) i k (t) is the probabiity that the k th automaton is in state i k at time t. Furthermore, the probabiity distribution of the N-dimensiona system, (t), is given by the tensor product of the probabiity vectors of the individua automaton at time t, i.e., (t) = NO k= (k) (t): (4) To sove N-dimensiona systems that are formed from independent stochastic automata is therefore very simpe. It suces to sove for the probabiity distributions of the individua stochastic automata and to form the tensor product of these distributions. Athough such systems may exist, the more usua case occurs when the transitions of one automaton may depend on the state of a second. It is to this topic that we now turn. 3.2 Interacting Stochastic Automata There are basicay two ways in which stochastic automata interact:. The rate at which a transition may occur in one automaton may be a function of the state of other automata. Such transitions are caed functiona transitions. 2. A transition in one automaton may force a transition to occur in one or more other automata. We aow for both the possibiity of a master/save reationship, in which an action in one automaton (the master) actuay occasions a transition in one or more other automata (the saves), and for the case of a rendez-vous in which the presence (or absence) of two or more automata in designated states causes (or prevents) transitions RR n2939

8 rigitte Pateau and Wiiam J. Stewart to occur. We refer to such transitions coectivey under the name of synchronized transitions. Synchronized transitions are triggered by a synchronizing event; indeed, a singe synchronizing event wi generay cause mutipe synchronized transitions. Synchronized transitions may aso be functiona. The eements in the matrix representation of any singe stochastic automaton are either constants, i.e., nonnegative rea numbers, or functions from the goba state space to the nonnegative reas. Transition rates that depend ony on the state of the automaton itsef, and not on the state of any other automaton, are to a intents and purposes, constant transition rates. A synchronizing transition may be either functiona or constant. In any given automaton, transitions that are not synchronizing transitions are said to be oca transitions. onsider as an exampe, a simpe queueing network consisting of two service centers in tandem and an arriva process that is Poisson at rate. Each service center consists of an innite queue and a singe server. The service time distribution of the rst server is assumed to be exponentia at xed rate, whie the service time distribution at the second is taken to be exponentia with a rate that varies with the number and distribution of customers in the network. Since a state of the network is competey described by the pair (n ; n 2 ) where n denotes the number of customers at station and n 2 the number at station 2, the service rate at station 2 is more propery written as (n ; n 2 ). We may dene two stochastic automata A () and A (2) corresponding to the two different service centers. The state space of each is given by the set of nonnegative integers f; ; 2; : : : ; g since any nonnegative number of customers may be in either station. It is apparent that the rst automaton A () is competey independent of the second. On the other hand, transitions in A (2) depend on the rst automaton in two ways. Firsty the rate at which customers are served in the second station depends on the number of customers in the network and hence, in particuar, on the number at the rst station. Thus A (2) contains functiona transition rates, ((n ; n 2 )). Secondy, when a departure occurs from the rst station, a customer enters the second and therefore instantaneousy forces a transition to occur within the second automaton. The state of the second automaton is instantaneousy changed from n 2 to n 2 +! This entais transitions of the second type, namey synchronizing transitions. Let us examine how these two dierent types of interaction may be specied in a stochastic automata network. onsider rst the case of constant and functiona transition rates on oca transitions; Normay an automaton wi contain both. The eements in the innitesima generator of any singe stochastic automaton are either constants, i.e., nonnegative rea numbers, or functions from the goba state space to the nonnegative reas. Transition rates that depend ony on the state of the automaton itsef, and not on the state of any other automaton, are to a intents and purposes, constant transition rates. This is INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 9 the case if the rate of transition of the second exponentia server in the two-station queueing network exampe is oad-dependent, i.e., depending ony on the number of customers present in station 2. The oad-dependent vaue is instantiated at each state and taken to be that constant vaue in the rest of the anaysis. onstant transition rates obviousy pose no probem. If a given state of a stochastic automaton occasions functiona transitions, the rate of these transitions must be evauated according to their dening formuae and the current goba state. In certain instances it may be advantageous to evauate these functiona rates ony once and to store the vaues obtained in an array from which they may be retrieved as and when they are needed. In other cases, it may be better to eave them in their origina form and to re-evauate the formua each time. Whether the transition rates are constant or functiona, it is important to note that ony the state of the oca automaton is aected. Therefore a the information concerning constant and functiona transition rates within an automaton can be handed within that automaton (assuming ony that that automaton has a knowedge of the goba system state). Functiona transitions aect the goba system ony in that they change the state of a singe automaton. Now consider synchronizing events. The two-station queueing network exampe given previousy ceary shows that the rst of the two stochastic automata is independent of the second. However, this does not mean that the compete set of information needed to specify the synchronizing event can be conned to A (2) unfortunatey. It is not sucient for A (2) to reaize that it is susceptibe to instantaneous transitions forced upon it by another automaton. Additionay A (), athough competey independent, needs to participate in a mechanism to dissimuate the fact that it executes synchronizing transitions. One way to impement this is to generate a ist of a possibe synchronizing events that can occur in a SAN. This ist needs to provide a unique name for each synchronizing event, the manner in which it occurs and its eect on other stochastic automata. In contrast to functiona transitions, synchronizing events aect the goba system by possiby atering the state of severa automata. 3.3 The Eect of Synchronizing Events We begin with a sma exampe of two interacting stochastic automata, A () and A (2), whose innitesima generator matrices are given by Q ()? = 2? 2 and Q (2) =?? 2 2 3? 3 respectivey. At the moment, neither contains synchronizing events nor functiona transition rates. The innitesima generator of the goba, two-dimensiona system is therefore given as The extension to more than two is immediate. A RR n2939

rigitte Pateau and Wiiam J. Stewart Q () Q (2) =?( + )?( + 2 ) 2 3?( + 3 ) 2?( 2 + ) 2?( 2 + 2 ) 2 2 3?( 2 + 3 ) : (5) A Its probabiity distribution vector (at time t) is obtained by forming the tensor product of the individua probabiity distribution vectors of each stochastic automata at time t. Let us now observe the eect of introducing synchronizing events. Suppose that each time automaton A () generates a transition from state 2 to state (at rate 2 ), it forces the second automaton into state. It may be readiy veried that the goba generator matrix is given by?( + )?( + 2 ) 2 3?( + 3 ) 2?( 2 + ) 2?( 2 + 2 ) 2 2 3?( 2 + 3 ) If, in addition, the second automaton A (2) initiates a synchronizing event each time it moves from state 3 to state (at rate 3 ), by for exampe forcing the rst automaton into state, we obtain the foowing goba generator.?( + )?( + 2 ) 2 3?( + 3 ) 2?( 2 + ) 2?( 2 + 2 ) 2 2 + 3?( 2 + 3 ) Our immediate reaction in observing these atered matrices may be to assume that a major disadvantage of incorporating synchronizing transitions is to remove the possibiity of representing the goba transition rate matrix as a (sum of) tensor products. However, Pateau [3] has shown that, by separating oca transitions from synchronizing transitions, this is not necessariy so; that the goba transition rate matrix can sti be written as a (sum of) tensor products. To observe this we proceed as foows. : A : A INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. The transitions at rates ; and 2 are not synchronizing events, but rather oca transitions. The part of the goba generator that consists uniquey of oca transitions may be obtained by forming the tensor sum of innitesima generators Q () and Q (2) that represent ony oca transitions, viz:. Q ()? = and Q (2) = A ; with tensor sum Q = Q () Q (2) =?? 2 2?( + )?( + 2 ) 2??? 2 2 The rates 2 and 3 are associated with two synchronizing events that we ca e and e 2 respectivey. The part of the goba generator that is due to the rst synchronizing event is given by Q e = 2? 2 2? 2 2? 2 which is the (ordinary) matrix sum of two tensor products, viz: Q e = A + 2? 2 A Simiary, the part of the goba generator due to synchronizing event e 2 is Q e2 = 3? 3 3? 3 which may be obtained from a sum of tensor products as Q e2 = 3 A + A? 3 A : A : : A RR n2939

2 rigitte Pateau and Wiiam J. Stewart Observe that the goba innitesima generator is now given by Q = Q + Q e + Q e2 : Athough we considered ony a simpe exampe, the above approach has been shown to be appicabe in genera. Stochastic automata networks that contain synchronizing transitions may aways be treated by separating out the oca transitions, handing these in the usua fashion by means of a tensor sum and then incorporating the sum of two additiona tensor products per synchronizing event. Furthermore, since tensor sums are dened in terms of the (usua) matrix sum of tensor products, the innitesima generator of a system containing N stochastic automata with E synchronizing events (and no functiona transition rates) may be written as 2E+N X j= N i= Q(i) j : (6) This quantity is referred to as the descriptor of the stochastic automata network. The computationa burden imposed by synchronizing events is now apparent and is twofod. Firsty, the number of terms in the descriptor is increased, two for each synchronizing event. We may therefore concude that the SAN approach is not we suited to modes in which there are many synchronizing events. On the other hand, it may sti be usefu for systems that may be modeed with severa stochastic automata that operate mosty independenty and ony infrequenty need to synchronize their operations, such as those found in many modes of highy parae machines. A second and even greater burden is that the simpe form of the soution, equation (4), no onger hods. Athough we have been successfu in writing the descriptor in a compact form as the sum of tensor products, the soution is not simpy the sum of the vectors computed as the tensor product of the soutions of the individua Q (i) j. Other methods for computing soutions must be found. The usefuness of the SAN approach wi be determined uniquey by our abiity to sove this probem. We now turn our attention to functiona transition rates, for these may appear not ony in oca transitions, but aso in synchronizing events. 3.4 The Eect of Functiona Transition Rates We return to the two origina automata given in equation (5) and consider what happens when one of the transition rates of the second automaton becomes a functiona transition rate. Suppose, for exampe, that the rate of transition from state 2 to state 3 in the second stochastic automaton is ^ 2 when the rst is in state and ~ 2 when the rst is in state 2. The goba innitesima generator is now INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 3?( + )?( + ^ 2 ) ^ 2 3?( + 3 ) 2?( 2 + ) 2?( 2 + ~ 2 ) ~ 2 2 3?( 2 + 3 ) If, in addition, the rate at which the rst stochastic automaton produces transition from state to state 2 is ; ^ and ~ depending on whether the second automaton is in state, 2 or 3, the two-dimensiona innitesima generator is given by?( + )?(^ + ^ 2 ) ^ 2 ^ 3?( ~ + 3 ) ~ 2?( 2 + ) 2?( 2 + ~ 2 ) ~ 2 2 3?( 2 + 3 ) However, it is sti possibe to prot from the fact that the nonzero structure is unchanged. This is essentiay what Pateau has done in her extension of the cassica tensor agebraic concepts, [33]. The descriptor is sti written as in equation (6), but now the eements of may be functions. This means that it is necessary to track eements that are functions and to substitute the appropriate numerica vaue each time the functiona rate is needed. A moment's reection shoud convince the reader that the introducion of functiona transition rates has no eect on the structure of the goba transition rate matrix other than when functions evauate to zero in which case a degenerate form of the origina structure is obtained. However, even if the structure is preserved, the actua vaues of the nonzero eements prevents us from writing the soution in the simpe form of equation (4). Nevertheess it is sti possibe to prot from this unatered nonzero structure. This is the concept behind the extended (generaized) tensor agebraic approach, [33]. The descriptor is sti Q (i) j written as in equation (6), but now the eements of Q (i) j may be functions. This means that it is necessary to track eements that are functions and to substitute (or recompute) the appropriate numerica vaue each time the functiona rate is needed. 4 Exampes We now introduce two fairy arge modes that we wi use for purposes of iustration. The rst is a mode of resource sharing that incudes functiona transitions. The second is a nite queueing network mode with both functiona transitions and synchronizing events. : A : A RR n2939

4 rigitte Pateau and Wiiam J. Stewart 4. A Mode of Resource Sharing In this mode, N distinguishabe processes share a certain resource. Each of these processes aternates between a seeping state and a resource using state. However, the number of processes that may concurrenty use the resource is imited to P where P N so that when a process wishing to move from the seeping state to the resource using state nds P processes aready using the resource, that process fais to access the resource and returns to the seeping state. Notice that when P = this mode reduces to the usua mutua excusion probem. When P = N a of the the processes are independent. Let (i) be the rate at which process i awakes from the seeping state wishing to access the resource, and et (i) be the rate at which this same process reeases the resource when it has possession of it. In our SAN representation, each process is modeed by a two state automaton A (i), the two states being seeping and using. We sha et sa (i) denote the current state of automaton A (i). Aso, we introduce the function f = NX i= (sa (i) = using) < P where (b) is an integer function that has the vaue if the booean b is true, and the vaue otherwise. Thus the function f has the vaue when access is permitted to the resource and has the vaue otherwise. Figure provides a graphica iustration of this mode.! ; A () A (N) seeping... seeping () (N) () f... (N) f using using Figure : Resource Sharing Mode The oca transition matrix for automaton A (i) is Q (i)? (i) f = (i) f ; (i)? (i) INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 5 and the overa descriptor for the mode is Q = M g N i= Q (i) = NX i= I 2 g g I 2 g Q (i) g I 2 g g I 2 ; where g denotes the generaized tensor operator, a precise denition of which is given in Section 5. The SAN product state space for this mode is of size 2 N. Notice that when P =, the reachabe state space is of size N +, which is consideraby smaer than the product state space, whie when P = N the reachabe state space is the entire product state space. Other vaues of P give rise to intermediate cases. 4.2 A Queueing Network with ocking and Priority Service The second mode we sha use is an open queueing network of three nite capacity queues and two customer casses. ass customers arrive from the exterior to queue according to a Poisson process with rate. Arriving customers are ost if they arrive and nd the buer fu. Simiary, cass 2 customers arrive from outside the network to queue 2, aso according to a Poisson process, but this time at rate 2 and they aso are ost if the buer at queue 2 is fu. The servers at queues and 2 provide exponentia service at rates and 2 respectivey. ustomers that have been served at either of these queues try to join queue 3. If queue 3 is fu, cass customers are bocked (bocking after service) and the server at queue must hat. This server cannot begin to serve another customer unti a sot becomes avaiabe in the buer of queue 3 and the bocked customer is transferred. On the other hand, when a (cass 2) customer has been served at queue 2 and nds the buer at queue 3 fu, that customer is ost. Queue 3 provides exponentia service at rate 3 to cass customers and at rate 32 to cass 2 customers. It is the ony queue to serve both casses. In this queue, cass customers have preemptive priority over cass 2 customers. ustomers departing after service at queue 3 eave the network. We sha et k?, k = ; 2; 3 denote the nite buer capacity at queue k. Queues and 2 can each be represented by a singe automaton (A () and A (2) respectivey) with a one-to-one correspondance between the number of customers in the queue and the state of the associated automaton. Queue 3 requires two automata for its representation; the rst, A (3), provides the number of cass customers and the second, A (32), the number of cass 2 customers present in queue 3. Figure 2 iustrates this mode. This SAN has two synchronizing events: the rst corresponds to the transfer of a cass customer from queue to queue 3 and the second, the transfer of a cass 2 customer from queue 2 to queue 3. These are synchronizing events since a change of state in automaton A () or A (2) occasioned by the departure of a customer, must be synchronized with a corresponding change in automaton A (3) or A (32), representing the arriva of that customer to queue 3. We sha denote these synchronizing events as s and s 2 respectivey. In addition to these synchronizing events, this SAN required two functions. They are: f = (sa (3) + sa (32) < 3? ) RR n2939

6 rigitte Pateau and Wiiam J. Stewart A () 2 oss? A (2) 2? A (3 ) A (3 2 ) 3? 3 3 2 2 oss oss Figure 2: Network of Queues Mode g = (sa (3) = ) The function f has the vaue when queue 3 is fu and the vaue otherwise, whie the function g has the vaue when a cass customer is present in queue 3, thereby preventing a cass 2 customer in this queue from receiving service. It has the vaue otherwise. Since there are two synchronizing events, each automaton wi give rise to ve separate matrices in our representation. For each automaton k we wi have a matrix of oca transitions, denoted by Q (k) Q (k) s ; a matrix corresponding to each of the two synchronizing events, and Q (k) s 2, and a diagona corrector matrix for each synchronizing event, Q (k) s and Q (k) s 2. In these ast two matrices, nonzero eements can appear ony aong the diagona; they are dened in such a way as to make k Q (k) s j + k Q (k) s j ; j = ; 2, generator matrices (row sums equa to zero). The ve matrices for each of the four automata in this SAN are as foows (where we use I m to denote the identity matrix of order m). For A () : Q () =??...........? A ; Q() s =........... A ; INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 7 For A (2) : Q (2) = For A (3) : Q (3) = For A (32) : Q (32) = Q () s =?...........??? 2 2? 2 2...........? 2 2 Q (2) s 2 = A ;? 2...........? 2? 2 3? 3........... 3? 3 3? 3 Q (3) s = f f........... f A ; 32 g? 32 g........... 32 g? 32 g 32 g? 32 g A ; A ; Q(2) s 2 = A ; A ; Q() s 2 = I = Q () s 2 : Q(3) s 2........... 2 2 Q(2) s = I 2 = Q (2) s : f f........... f Q(3) s 2 = I 3 = Q (3 ) s 2 : Q(32) s 2 Q (32) s 2 = I 3 = Q (32) s = Q (3 2) s : A ; A ;? f f? f f...........? f f A ; RR n2939

8 rigitte Pateau and Wiiam J. Stewart The overa descriptor for this mode is given by Q = M g Q(i) + O g Q(i) s + O g Q (i) s + O g Q(i) s 2 + O g where the generaized tensor sum and the four generaized tensor products are taken over the index set f, 2, 3 and 3 2 g. The reachabe state space of the SAN is of size 2 3 ( 3 +)=2 whereas the compete SAN product state space has size 2 3 2. Finay, we woud ike to draw our readers attention to the sparsity of the matrices presented above. 5 Product Forms Stochastic automata networks constitute a genera modeing technique and as such they can sometimes inherit resuts from those aready obtained by other modeing approaches. Jackson networks, for exampe, may be represented by a SAN; the reversibiity resuts of Key, [26], and the competition conditions of oucherie, [6], can be appied to SANs eading to product forms. In this section we sha present sucient conditions for a SAN to have a product form soution. These conditions extend those given by oucherie and in addition may be shown to be appicabe to truncated state spaces. They appy ony to SANs with no synchronizing events, which means that the transitions of the SAN can ony be transitions of one automaton at a time. Thus Jackson networks ie outside their scope of appicabiity. The way we proceed is to work on goba baance equations and search for sucient conditions on the functiona transition rates to obtain a product form soution. Let us state the probem more formay. onsider a SAN with N automata and oca transition matrices Q (k), k = ; 2; : : : ; N. The states of A (k) are denoted i k 2 S (k), and a state of the SAN is denoted i = (i ; :::; i N ). A state of the SAN without automaton A (k) is denoted ik = (i ; :::; i k? ; i k+ ; :::; i N ). A state i in which the k th component is repaced by i k is denoted by ik ji k. The eement of Q(k) are assumed to be of product form type; i.e., for i k 6= i k, Q (k) (i k ; i k) = q (k) (i k ; i k)f (k) (i; i ); where q (k) (i k ; i k ) is a constant transition rate and f (k) (i; i ) is any positive function. This formuation does not restrict the cass of SAN with functiona transition rates. Notice that when the transition of the SAN is occasioned by a transition of its k th automaton, the function f (k) (i; i ) actuay depends ony on i and i k. Assume now that the q (k) (i k ; i k ) satisfy baance equations in the sense that there exist Q (i) s 2 ; a set of positive numbers (k) (i k ) which sum to and, for a i k 2 S (k), satisfy The SAN generator is Q = L N X (k) (i k )q (k) (i k ; i k)? (k) (i k)q (k) (i k; i k ) i k 2S(k) k= Q(k) = : (7). Its transition rates, for i 6= i, are given by q(i; i ) = q (k) (i k ; i k )f (k) (i; ik ji k ) if i = ik ji k otherwise INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 9 The reachabe state space of the SAN is denoted by R Q and, because of the eect of functiona N transitiona rates, can be stricty smaer that S = S(k). The goba baance equations for the SAN are, for i 2 R, X i 2S Substituting for q(i; i ) and q(i ; i) in (8) yieds NX X k= i k 2S(k) ((i)q(i; i )? (i )q(i ; i)) = : (8) (i)q (k) (i k ; i k)f (k) (i; ik ji k)? ( ik ji k)q (k) (i k; i k )f (k) ( ik ji k; i) = : (9) This SAN has a product form soution if, for some normaizing constant, (i) = Q N (k) (i k ) is a soution of these baance equations. Substituting this into (9) gives NX NY k= j=;j6=k (j) (i j ) NX i k 2S(k) (k) (i k )q (k) (i k ; i k)f (k) (i; ik ji k)? (k) (i k)q (k) (i k ; i k)f (k) ( ik ji k ; i) = : Now it ony remains to nd sucient conditions on the functions f (k) for which NX (k) (i k )q (k) (i k ; i k)f (k) (i; ik ji k)? (k) (i k)q (k) (i k ; i k)f (k) ( ik ji k ; i) i k 2S(k) () is equa to zero, knowing the oca baance equations, (7). First case: The functions f (k) express a truncation of the state space of the SAN (simiar to that described by Key). That is to say, the functions are equa to the indicator function of the reachabe state space R: f (k) (i; i ) = (i 2 R). Thus the expression () is triviay equa to zero: either ik ji k 2 R (the functions are equa to ) and we have the oca baance equations, or ik ji k =2 R and the functions themseves are zero. The normaizing constant (i) and might be dicut to compute if R is arge. is the inverse of P i2r Q N k= Second case: The functions f (k) depend ony on ik and not on the current state of automaton k. This means that the decomposition Q (k) (i k ; i k ) = q(k) (i k ; i k )f (k) (i; i ) is a rea product form. The variabe q (k) (i k ; i k ) is the oca transition rate of automaton k and f (k) (i; i ) = f (k) ( ik ) expresses the interaction of the rest of the SAN when it is in state ik. In essence, the functions f (k) ( ik ) either force the system to hat, if they evauate to zero, or ese permit the automaton A (k) to execute independenty, abeit with modied rates: the function uniformy sows down or speeds up the automata for a given ik. When ik changes, the sowing/speeding factor changes. The baance equations are given by f (k) ( ik ) NY j=;j6=k (j) (i j ) X i k 2S(k) (k) (i k )q (k) (i k ; i k)? (k) (i k)q (k) (i k ; i k) = RR n2939

2 rigitte Pateau and Wiiam J. Stewart which must hod because the oca baance equations themseves, (7), hod. This second case is a generaization of the oucherie competition conditions. The constant is equa to when the reachabiity space is the product space; otherwise it must be chosen so that the individua probabiities sum to one. Third case: Notice that the two previous cases are not overapping, and this for two reasons: In case, f (k) (i; i ) = (i 2 R) = ( ik ji k 2 R). In genera, the function ( ik ji k 2 R) depends not ony on ik, but on i k as we. In case 2, we have a \uniform" modication of A (k) either or unchanged, for a given ik. rates whie in case, they are This presents the possibiity of combining cases and 2 to yied a third case: f (k) (i; i ) = (i 2 R)f (k) ( ik ): Using the notation described above, we may summarize these resuts in the foowing theorem: Theorem 5. Given a SAN having no synchronizing events and in which the eements of Q (k) are of product form type; i.e., for i k 6= i k, Q (k) (i k ; i k) = q (k) (i k ; i k)f (k) (i; i ); each of the foowing sucient conditions eads to a product form soution, (i) = Q N (k) (i k ): ase : f (k) (i; i ) = (i 2 R) ase 2: f (k) (i; i ) = f (k) ( ik ) ase 3: f (k) (i; i ) = (i 2 R)f (k) ( ik ) The (i) satisfy the goba baance equations for the SAN, is a normaizing constant and the (k) (i k ) are soutions of the oca baance equations, (7). Exampes:. The resource sharing exampe of Section 4. fas into case with f (k) (i; i ) = N X (i j = using) < P A = N X j=;j6=k j=;j6=k (i j = using) < PA : INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 2 2. onsider a number, N, of identica processes each represented by a three state automaton. State represents the state in which the process computes independenty; state 2 represents an interacting state (int) in which the automaton computes and sends messages to the other automata and state 3 is a state in which the process has excusive access in order to write (w) to a critica resource. Each process may move from any of the three states to any other according to we dened rates of transition. To provide for mutuay excusive access to the critica resource, the rates of transition to state 3 must be mutipied by a function g dened as g(i; i ) = X N j= (i j = w) = A : To provide for the eect of communication overhead, a transition rates within the automaton k; k = ; 2; : : : ; N must be mutipied by a function f (k) dened as f (k) (i; i ) = P N j=;k6=j (i j = int) : Such a SAN therefore incorporates a superposition of cases and 2. 3. The exampes provided in the paper of oucherie, [6]; viz: the dining phiosophers probem, ocking in a database system, and so on, a fa into case 2. In these exampes, the functions f (k) ( ik ) express a reachabe state space and yied a uniform mutipicative factor. Other exampes may be found in [4, 9, 22, 28]. 4. Our na exampe expicity dispays the dependence of the function on i and fas into case. onsider a system consisting of P units of resource and N identica processes, each represented by a three-state Markov chain. Whie in state, a process may be considered to be seeping; in state, it uses a singe unit of resource; whie in state 2, the process uses 2 units of the resource. The transitions among its three states are such that whie in the seeping state (), it can move directy to either state or 2, (i.e., it may request, and receive, two units of resource, or just a singe unit of resource). From state 2, the process can ony move to state, and from state it can ony move to state, (i.e., units of the resource are reeased independenty). To cater for the case in which sucient resources are not avaiabe, the rates of transition towards states and 2 must each be mutipied by the function f (k) (i; i k) = i k + NX j=;j6=k 6 Vector-Descriptor Mutipications i j < PA : In many cases, and perhaps most, product form soutions are not avaiabe and the anayst must turn to other soutions procedures. When the goba innitesima generator of a SAN RR n2939

22 rigitte Pateau and Wiiam J. Stewart is avaiabe ony in the form of a SAN descriptor, the most genera and suitabe methods for obtaining probabiity distributions are numerica iterative methods, [44]. Thus, the underying operation, whether we wish to compute the stationary distribution, or the transient soution at any time t, is the product of a vector with a matrix. Since the basic operation is X 2E+N xq = x j= X 2E+N N i= Q(i) j = x N i= Q (i) j= x N i= Q(i) j ; where, for notationa convenience, we have removed the subscripts on the Q (i) j. It is essentia that this operation be impemented as ecienty as possibe. The foowing theorem is proven in [33]. It impicity assumes that the SAN matrices are dense. Theorem 6. The product x N i= Q (i) ; where Q (i), of order n i, contains ony constant terms and x is a rea vector of ength Q N i= n i, may be computed in N mutipications, where N = n N ( N? + NY i= n i ) = Let us now examine the eect of introducing functiona rates. The savings made in the computation of x N i= Q(i) are due to the fact that once a product is formed, it may be used in severa paces without having to re-do the mutipication. With functiona rates, the eements in the matrices may change according to their context so that this same savings is sometimes possibe [44]. This eads to an extension of some of the properties of tensor products and to the concept of Generaized Tensor Products (GTPs) as opposed to Ordinary Tensor Products (OTP). 7 Generaized Tensor Products We assume throughout that a matrices are square. As indicated in the previous section, [A] indicates that the matrix may contain transitions that are a function of the state of the automaton A. More generay, A (m) [A () ; A (2) ; : : : ; A (m?) ] indicates that the matrix A (m) may contain eements that are a function of one or more of the states of the automata A () ; A (2) ; : : : ; A (m?). We sha use the notation g to denote a generaized tensor product. Thus A g [A] denotes the generaized tensor product of the matrix A with the functiona NY i= n i NX i= n i : INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 23 matrix [A] and we have A g [A] = a (a ) a 2 (a ) a na (a ) a 2 (a 2 ) a 22 (a 2 ) a 2na (a 2 )...... a na(a na ) a na2(a na ) a nana (a na ) A ; () where (a k ) represents the matrix when its functiona entries are evauated with the argument a k ; k = ; 2; : : : ; n a, the a i being the states of automaton A. Aso, where A[] g = a []I nb a 2 []I nb a na []I nb a 2 []I nb a 22 []I nb a 2na []I nb...... a na[]i nb a na2[]i nb a nana []I nb a ij []I nb = diagfa ij (b ); a ij (b 2 ); : : : ; a ij (b nb )g and a ij (b k ) is the vaue of the ij th eement of the matrix A when its functiona entries are evauated with the argument b k ; k = ; 2; : : : ; n b. Finay, when both automata are functiona we have A[] g [A] = A ; a []I nb (a ) a 2 []I nb (a ) a na []I nb (a ) a 2 []I nb (a 2 ) a 22 []I nb (a 2 ) a 2na []I nb (a 2 )...... a na[]i nb (a na ) a na2[]i nb (a na ) a nana []I nb (a na ) For A[] g [A], the generic entry (; k) within bock (i; j) is a ij (b k ) b k (a i ). We now present a number of emmas concerning generaized tensor products. Their proofs may be found in [5]. These emmas are usefu for deriving many important properties of generaized tensor products incuding Theorems 7. and 7.2 which foow and which specify the compexity of forming the product of a vector with a SAN descriptor in the presence of functiona transitions. Lemma 7. (GTP: Associativity) (A[; ] g [A; ]) g [A; ] = A[; ] g ([A; ] g [A; ]) Lemma 7.2 (GTP: Distributivity over Addition) (A [] + A 2 []) g ( [A] + 2 [A]) = (A [] g [A] + A [] g 2 [A] + A 2 [] g [A] + A [] g 2 [A]) A : RR n2939

24 rigitte Pateau and Wiiam J. Stewart As for ordinary tensor products, compatibiity with mutipication usuay does not hod for generaized tensor products either. However, there exists three degenerate compatibiity forms when some of the factors are identity matrices and not a of the factors have functiona entries. They are Lemma 7.3 (GTP: ompatibiity over Mutipication: I Two Factors) (A[] []) g I nc = (A[] g I nc ) ([] g I nc ) Simiary, I nc g (A[] []) = (I nc g A[]) (I nc g []): Lemma 7.4 (GTP: ompatibiity over Mutipication: II Two Factors) A g [A] = [A I na ] g [I nb [A]] = (I na g [A]) (A I nb ) : Lemma 7.5 (GTP: ompatibiity over Mutipication: III Two Factors) A[] g = [A[] I na ] g [I nb ] = (A[] g I nb ) (I na ) : This emma sti hods if I na is repaced by any constant matrix. Lemma 7.6 (GTP: ompatibiity over Mutipication Many Factors) A () g A (2) [A () ] g A (3) [A () ; A (2) ] g g A (m) [A () ; : : : ; A (m?) ] = I :m? g A (m) [A () ; : : : ; A (m?) ] I :m?2 g A (m?) [A () ; : : : ; A (m?2) ] g I m:m I : g A (2) [A () ] g I 3:m A () g I 2:m (2) Due to the existence of Lemma 7.5 the same property hods for A (m) [A () ; : : : ; A (m?) ] g A (m?) [A () ; : : : ; A (m?2) ] g g A (2) [A () ] g A () = A (m) [A () ; : : : ; A (m?) ] g I m?: I m:m g A (m?) [A () ; : : : ; A (m?2) ] g I m?2: I m:3 g A (2) [A () ] g I : I m:2 g A () (3) INRIA

Stochastic Automata Networks: Product Forms and Iterative Soutions. 25 In Lemma 7.6, ony one automaton can depend on a the (m? ) other automata, ony one can depend on at most (m? 2) other automata and so on. One automaton must be independent of a the others. This provides a means by which the individua factors on the eft-hand side of equation (2) may be ranked; i.e., according to the number of automata on which they may depend. An automaton may actuay depend on a subset of the automata in its parameter ist. Lemma 7.7 (GTP: Pseudo-ommutativity) Let be a permutation Q of the integers N [; 2; : : : ; N], then there exists a permutation matrix, P of order n i= i, such that ON O A (k) [A () ; : : : ; A (N) N ] = P A ((k)) [A () ; : : : ; A (N) ]P T : g k= g k= These emmas aow the foowing theorem to be proven. (The proof itsef may be found in [5].) Theorem 7. (GTP: Agorithm) The mutipication x A () g A (2) [A () ] g A (3) [A () ; A (2) ] g g A (N) [A () ; : : : ; A (N?) ] where x is a rea vector of ength Q N i= n i may be computed in O( N ) mutipications, where N = n N ( N? + NY i= n i ) = NY i= n i NX i= n i : This compexity resut was computed under the assumption that the matrices are fu. However, the number of mutipications may be reduced by taking advantage of the fact that the bock matrices are generay sparse. It is immediatey apparent that when the matrices are not fu, but possess a specia structure such as tridiagona, or contain ony one nonzero row or coumn, etc., or are sparse, then this may be taken into account and this number reduced in consequence. The cost of the function evauations is incuded in the denition of the big Oh formua. We woud ike to point out that athough Theorem 7. aows us to reorganize the terms in the generaized tensor product in any way we wish, the advantage of eaving them in the form given above is precisey that the computation of the state indices can be moved outside the innermost summation of the agorithm. The foowing agorithm is based directy on Theorem 7. and impements an ecient product of a vector x with a generaized tensor product in which the automata satisfy the functiona dependencies described in Lemma 7.6. In this agorithm, the notation A (i) [a () k ; : : : ; a (i?) k i? ] impies that the matrix is evauated under the assumption that automaton A (j) is in state a (j) k j, for j = ; 2; : : : ; i?. RR n2939