SPATIAL pattern formation is central to the understanding

Transcription

1 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 1 A Forma Methods Approach to Pattern Recognition and Synthesis in Reaction Diffusion Networks Ezio Bartocci, Ebru Aydin Go, Member, IEEE, Iman Haghighi, Student Member, IEEE, and Cain Beta, Senior Member, IEEE Abstract We introduce a forma framework for specifying, detecting and generating spatia patterns in reaction-diffusion networks. Our approach is based on a nove spatia superposition ogic, whose semantics is defined over the quad-tree representation of a partitioned image. We demonstrate how to use ruebased cassifiers to efficienty earn spatia superposition ogic formuas for severa types of patterns from positive and negative exampes. We impement pattern detection as a mode checking agorithm and we show that it achieves very good resuts on test data sets which are different from the training sets. We provide a quantitative semantics for our ogic and we deveop a computationa framework where our quantitative mode checking agorithm works in synergy with a partice swarm optimization technique to synthesize the parameters eading to the formation of desired patterns in reaction-diffusion networks. Index Terms Pattern recognition and formation, forma verification and synthesis, reaction-diffusion networks. I. INTRODUCTION SPATIAL pattern formation is centra to the understanding of how compex organisms deveop and how seforganization arises out of ocay interacting dynamica systems. Exampes of spatia patterns are ubiquitous in nature: from the stripes of a zebra and the spots on a eopard to the fiaments (Anabaena) [1], squares (Thiopedia rosea), and vortex (Paenibacius) [2] formed by singe-ce organisms. Pattern formation is not ony at the very origin of morphogenesis and deveopmenta bioogy, but it is aso at the core of technoogies such as sef-assemby, tissue engineering, and amorphous computing. Even though the study of spatia patterns has kinded the interest of severa communities such as bioogy, computer science, and physics, the mechanisms responsibe for their formation are not yet we understood. Pattern recognition is usuay considered as a branch of machine earning [3] where patterns have a statistica characterization [4] or they are described through a structura reationship among their features [5]. Even though pattern recognition has been successfu in severa appication areas [6], it sti acks of a forma foundation and a suitabe higheve specification anguage that can be used to specify patterns This work was partiay supported by ONR under grant ONR N and NSF under grant CBET E.B. acknowedge the support of the Austrian FFG project HARMONIA (nr ), the Austrian FWFfunded (nr. S N23) SHiNE project, the EU ICT COST Action IC1402 on Runtime Verification beyond Monitoring (ARVI). E. Bartocci is with the Institute of Computer Engineering, Vienna University of Technoogy, Vienna, Austria, ezio.bartocci@tuwien.ac.at. E. Aydin Go is with the Department of Computer Engineering, Midde East Technica University, Ankara, Turkey, ebrugo@metu.edu.tr. I. Haghighi and C. Beta are with the Division of Systems Engineering, Boston University, Boston, MA, USA, {haghighi,cbeta}@bu.edu. in a concise and intuitive way and to reason about them in a systematic way. In particuar we are interested in the foowing questions. Can patterns be specified in a forma anguage with wedefined syntax and semantics? Can we deveop agorithms for pattern detection from specifications given in such a anguage? Given a arge coection of ocay interacting agents, can we design parameter synthesis rues, contro and interaction strategies guaranteeing the emergence of goba patterns? In this paper, our goa is to provide some preiminary answers to such questions by drawing inspiration from the fied of computer aided verification and mode checking [7], [8]. We address the foowing probem: Given a network of ocay interacting dynamica systems, and given sets of positive and negative exampes of a desired pattern, find parameter vaues that guarantee the occurrence of the pattern in the network at steady state. Our approach everages on a nove spatia superposition ogic, caed Tree Spatia Superposition Logic (TSSL), whose semantics are defined over quad-trees of partitioned images. In our setting, a pattern descriptor is a TSSL formua and detecting the existence of a pattern in an image is a mode checking probem. We can either manuay specify the pattern using the TSSL syntax or we can empoy machine-earning techniques using rue-based cassifiers to infer such a formua from given sets of positive and negative exampes. We aso deveop a computationa framework where our mode checking agorithm works in synergy with a partice swarm optimization technique to synthesize the parameters eading to patterns of interest in reaction-diffusion systems. The optimization fitness function is given by a measure of satisfaction induced by the quantitative semantics that we define for the ogic. The positive and negative signs of this measure are sound w.r.t. the satisfaction or vioation of the formua, whie the absoute vaue represents how far an image is from a desired pattern. We provide exampes demonstrating that TSSL formuas can encode, for some commony encountered patterns, very good cassifiers. Furthermore, we compared TSSL formuas with traditiona inear cassifiers, and in a of the exampes the cassification rate of the TSSL formua was the highest (over 95%). In the exampes, we focus on the Turing reaction-diffusion network [9], and show that pattern-producing parameters can be automaticay generated with our method. However, the overa computationa approach can, in principe, be appied to any network of ocay interacting systems. The rest of the paper is organized as foows. In Section II we discuss reated work. In Section III we formuate the (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

2 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 2 probem and outine our approach. We define the syntax and semantics of TSSL in Section IV. A machine earning technique to earn TSSL formuas from positive and negative exampes of desired patterns is deveoped in Section V. The soution to the pattern generation probem is presented in Section VI as a supervised, iterative procedure that integrates quantitative mode checking and optimization. We concude with fina remarks and directions for future work in Section IX. II. RELATED WORK Pattern recognition is a we-estabished technique in machine earning. Given a data set and a set of casses, the goa is to assign each data to one cass, or to provide a most ikey matching of the data to the casses. The two main steps in pattern recognition are: to extract distinctive features [10] [13] with reevant information from a set of input data representing the pattern of interest and to buid, using one of the severa avaiabe machine earning techniques [14], an accurate cassifier trained with the extracted features. The descriptor chosen in the feature extraction phase depends on the appication domain and the specific probem. This work is reated to pattern recognition in computer vision, where these descriptors may assume different forms. Feature descriptors such as Textons [10] and Histograms of Oriented Gradients (HoG) [11] are concerned with statistica information of coor distributions of intensity gradients and edge directions. The scae-invariant feature transform (SIFT), proposed by Lowe in [13], is based on the appearance of an object at particuar interest points, and is invariant to image scae and rotation. The shape context [12] is another feature descriptor intended to describe the shape of an object by the points of its contours and the surrounding context. In this paper we estabish a connection between verification and pattern recognition. Both cassica verification [15] [19] and pattern recognition techniques aim to verify (and possiby quantify) the emergence of a behaviora pattern. We propose ogic formuas as pattern descriptors and verification techniques as pattern cassifiers. The ogica nature of such descriptors aows to reason about patterns and to infer interesting properties, such as spatia periodicity and sef-simiar (fracta) texture. Furthermore, combining different pattern descriptors using both moda and ogica operators is quite intuitive. This paper is inspired by the origina work on morphogenesis by Aan Turing [9], and is cosey reated to [20]. In the atter, the authors introduced a Linear Spatia Superposition Logic (LSSL), whose formuas were interpreted over quadtree image partitions. The existence of a pattern in an image corresponded to the existence of a path in the corresponding tree from the root to the eaf corresponding to a representative point in the image. As a consequence, the method was shown to work for spiras, for which the center was chosen as the representative point. The ogic proposed here is more genera as it does not depend on the choice of such a point and captures the pattern gobay. For exampe, the patterns considered in this paper cannot be expressed in LSSL, because they rey on a tree representation rather than a path representation. As opposed to [20], we aso define a quantitative semantics for the ogic, which can be seen as a distance to satisfaction given an image and a formua. We use this distance as a fitness function in an optimization probem to search for patternproducing parameters in a system. This quantitative semantics and the discounted mode checking on a computationa tree are inspired from [21], with the notabe difference that we do not need a metric distance, but rather a measure of satisfiabiity. Such measures have aso been used in [15] [19]. Whie such measures exist for cassica cassifiers such as Support Vector Machines (SVM) [3], Fisher Linear Discriminants (FLD) [3], and Kozinec s hyperpane [22], in the form of the distance from an image to the cassifying hyperpane in the feature space, we show (through numerica experiments) that the measure induced by the quantitative semantics of TSSL is better suited for optimization agorithms. This paper is aso reated to the vast iterature on consensus protocos (see [23] [25]). As in these works, here we consider a network of ocay interacting dynamica systems, and we are interested in achieving a desired, emergent goba behavior. However, as opposed to most works in this area, the goba behavior we consider is a spatiay-distributed pattern, rather than an agreement on some quantity. Moreover, rather than showing that some goba behavior emerges from given oca interactions, we design a top-down approach in which we prescribe the goba behavior and then synthesize the oca dynamics achieving it. Part of the materia from this paper appeared in the Proceedings of the IEEE Conference on Decision and Contro (CDC) 2014 [26], where most of the theoretica resuts were presented without proofs. In addition to the technica detais, this paper incudes: a notion of max distance between two quad transition systems (see Definition 9) in Section IV; a theorem on the correctness of the TSSL quaitative semantics w.r.t. the quantitative semantics given two quad transition systems with a given max distance (Theorem 2) in Section IV; a comparison of the cassification and quantification capabiities offered using TSSL w.r.t. the traditiona inear cassifiers in Section VII; a new version of TSSL with basic propositions expressing constraints over higher statistica moments and an exampe on the improved effectiveness for pattern synthesis in Section VIII. III. PROBLEM FORMULATION Notation. We use R, R +, N and N + to denote the set of rea numbers, non-negative reas, integer numbers, and nonnegative integers, respectivey. For any c R and set S R, S >c := {x S x > c}, and for any a,b R, S [a,b] := {x S a x b}. A reaction-diffusion network S is modeed as a spatiay distributed and ocay interacting K K rectanguar grid of identica systems, where each ocation (i, j) N [1,K] N [1,K] corresponds to a system: S i, j : dx(n) i, j dt = D n (u (n) i, j x(n) i, j ) + f n(x i, j,r), n = 1,...,N, (1) (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

3 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 3 where x i, j = [x (1) i, j,...,x(n) i, j ] is the N-dimensiona state vector of system S i, j, which captures the concentrations of a species of interest. Diffusion coefficients D = [D 1,...,D N ] R N + and reaction constants R R P N are the parameters of a system S. The oca dynamics f n : R N + R P N R are defined by R for each of the species n = 1,...,N. Note that the parameters and dynamics are the same for a systems S i, j,(i, j) N [1,K] N [1,K]. The diffusion coefficient is stricty positive for diffusibe species and it is 0 for non-diffusibe species. Finay, u i, j = [u (1) i, j,...,u(n) i, j ] is the input of system S i, j from the neighboring systems: u (n) i, j = 1 ν i, j x v (n), v ν i, j ν i, j denotes the set of indices of systems adjacent to S i, j. Given a parameter vector p = [D,R] R P, we use S (p) to denote an instantiation of a reaction-diffusion network. We use x(t) R K K N + to denote the state of system S (p) at time t, and x i, j (t) R N + to denote the state of system S (p) i, j at time t. Whie the mode captures the dynamics of concentrations of a species of interest, we assume that a subset {n 1,...,n o } {1,...,N} of the species is observabe through: H : R K K N + R K K o [0,b] : y = H(x), for some b R +. For exampe, a subset of the genes in a gene network are tagged with fuorescent reporters. The reative concentrations of the corresponding proteins can be inferred by using fuorescence microscopy. We are interested in anayzing the observations generated by system (1) in steady state. Therefore, we focus on parameters that generate steady state behavior, which can be easiy checked through a running average: K i=1 K N j=1 n=1 x (n) i, j (t) x(n) i, j < ε, (2) where x (n) i, j = t t T x(n) i, j (τ)dτ/t for a sufficienty arge T t. The system is said to be in steady state at time t, if (2) hods for a t t. In the rest of the paper, we wi simpy ca the observation of a trajectory at steady state as the observation of the trajectory, and denote it as H(x( t)). Exampe 1. We consider a reaction-diffusion network with two species (i.e. K = 32, N = 2): dx (1) i, j dt dx (2) i, j ( ) = D 1 u (1) i, j x(1) i, j + R 1 x (1) i, j x(2) i, j x(1) i, j + R 2, ( ) = D 2 u (2) i, j x(2) i, j + R 3 x (1) i, j x(2) i, j + R 4. (3) dt The system is inspired from Turing s reaction-diffusion system, which is presented in [27] as a mode of the skin pigments of an anima. At a ce (ocation (i, j)), the concentration of species 1, x (1) i, j, depends on the concentration of species 1 in this ce and in its neighbors (if D 1 > 0), and the concentration of species 2 in this ce ony, i.e. x (2) i, j. Simiary, x(2) i, j depends on the concentration of species 2 in this ce and in its neighbors (if D 2 > 0), and x (1) i, j (if R 3 0). We assume that species 1 is observabe through the mapping H : R R [0,1] : y = H(x), where y i, j = x (1) i, j max m,n x (1). m,n We simuate the system from random initia conditions with parameters R = [1, 12, 1,16], and different diffusion parameters D 1 = [5.6,24.5], D 2 = [0.2,20], and D 3 = [1.4,5.3]. The observed concentrations of species 1 at different time points are shown in Figure 1. At time t = 50, a trajectories are in steady state. Note that, in a three cases, the spatia distribution of the steady state concentrations of species 1 has some reguarity, i.e. it forms a pattern. We wi use arge spots (LS), fine patches (FP), and sma spots (SS) to refer to the patterns corresponding to D 1, D 2, and D 3, respectivey. (c) t=0 t=5 t=10 t=20 t=30 t=40 t=50 t=60 Fig. 1. Observations generated by system (3) with parameters R and D 1, D 2, and (c) D 3 from Exampe 1 (the concentration of species 1 is represented with shades of red). The steady state observations produce arge spots (LS), fine patches (FP), and (c) sma spots (SS). In this paper, we consider the foowing probem: Probem 1. Given a reaction-diffusion network S as defined in (1), a finite set of initia conditions X 0 R K K N, ranges of the design parameters P = P 1... P P, P i R,i = 1,...,P, a set of steady state observations Y + = {y i } i=1,...,n+ that contain a desired pattern, a set of steady state observations Y = {y i } i=1,...,n that do not contain the pattern, find parameters p P such that the trajectories of system S (p ) originating from X 0 are guaranteed to produce observations that contain the desired pattern. To sove Probem 1, we need to perform two steps: Design a mechanism that decides whether an observation contains a pattern. Deveop a search agorithm over the state space of the design parameters to find p. The first step requires the definition of a pattern descriptor. To this goa, we deveop a new spatia ogic over spatiasuperposition trees obtained from the observations, and treat the decision probem as a mode checking probem. The new ogic and the superposition trees are expained in Section IV. Then, finding a pattern descriptor reduces to finding a formua of the new ogic that specifies the desired pattern. We empoy machine-earning techniques to earn such a formua from the given sets of observations Y + and Y. The second step is the synthesis of parameters p such that the observations produced by the corresponding reactiondiffusion network S (p ) satisfy the formua earned in the first step. To this end, we introduce quantitative semantics for the new ogic, which assigns a positive vauation ony to the superposition-trees that satisfy the formua. This quantitative vauation is treated as a measure of satisfaction, and is used as the fitness function in a partice swarm optimization (PSO) (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

4 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 4 agorithm. The choice of PSO is motivated by its inherent distributed nature, and its abiity to operate on irreguar search spaces, i.e. it does not require a differentiabe fitness function. Finay, we propose a supervised, iterative procedure to find p that soves Probem 1. The procedure invoves iterative appications of steps one and two, and an update of the set Y unti a parameter set that soves Probem 1 is found, such that the corresponding steady state observations match the desired patterns defined by the user. IV. TREE SPATIAL SUPERPOSITION LOGIC A. Quad-tree spatia representation We represent the observations of a reaction-diffusion network as a matrix A k,k of 2 k 2 k eements a i, j with k N >0. Each eement corresponds to a sma region in the space and is defined as a tupe a i, j = a (1) i, j,,a(o) i, j of vaues representing the concentration of the observabe species within an interva a (c) i, j [0,b], with b R +. Given a matrix A k,k, we use A k,k [i s,i e ; j s, j e ] to denote the sub-matrix formed by seecting the rows with indices from i s to i e and the coumns with indices from j s to j e. Definition 1. A quad-tree Q = (V,R) is a quaternary tree [28] representation of A k,k where each vertex v V represents a sub-matrix of A k,k and the reation R V V defines the four chidren of each node v that is not a eaf. A vertex v is a eaf when a the eements of the sub-matrix that it represents have the same vaues. Figure 2 shows an exampe of a quadtree, where node v 0 represents the entire matrix; chid v 1 represents the submatrix {1,,2 k 1 } {1,,2 k 1 }; chid v 7 represents the sub-matrix {2 k 2 +1,,2 k 1 } {2 k 2 +1,,2 k 1 }; etc. In Figure 2, we aso abe each edge in the quad-tree with the direction of the sub-matrix represented by the chid: north west (NW), north east (NE), south west (SW), south east (SE). NW$ NW$ v 5 NE$ v 8 v 0 v 0 v 1 v NW$ 4 SW$ SE$ NE$ v 6 v 7 SW$ SE$ v 1 v 2 v 3 v 4 SW$ v 2 v 3 $ NE$ SE$ NW$ SW$ SE$ NE$ v 5 v 6 v 7 v 8 Fig. 2. Quad-tree representation of a matrix. Definition 2. We define the mean function µ c : V [0,b] for sub-matrix A k,k [i s,i e ; j s, j e ] represented by the vertex v V of the quad-tree Q = (V,R) as foows: µ c (v) = 1 (i e i s + 1)( j e j s + 1) a (c) i, j i, j {i s,,i e } { j s,, j e } The function µ c is the sampe mean and an estimation for the expected vaue for an observabe variabe with index c,1 c o in a particuar region of the space represented by the vertex v. Definition 3. Two vertices v a,v b V are said to be equivaent when the mean function appied to the eements of the submatrices that they represent produce the same vaues: $ Agorithm BUILDINGQUADTRANSITIONSYSTEM Input: Matrix A k,k of 2 k 2 k of eements a i, j = a (1) i, j,,a(o) i, j, its quad-tree Q=(V,R), the root v 0 V, and a abeing function LQ : R D = {NW,NE,SE,SW} Output: Quad Transition System Q T S = (S,s ι,τ,σ,[.],l) 1: Σ := {m 1,,m o } Initiaize the set of variabes Σ of Q T S. 2: τ = /0 Initiaize the set τ of the transition reation τ of Q T S. 3: S := {s ι } Initiaize the set of states S of Q T S. 4: T S := { s ι,{v 0 } } Each tupe in TS contains a state in S and a set of vertices in V. 5: LF := {v V t V : (v,t) R} LF is the set of eaves of Q 6: PLF := {P i LF,1 i n P i /0 v a,v b P i, v c P j i,v a v b v a v c } PLF is a partition of LF with equivaent eaves. 7: for each ˆP PLF do For each partition eement, create a state s with a sef-oop and a transition to the state s ι if ˆP contains a chid of v 0. 8: add new state s to S and a tupe s, ˆP to T S 9: τ := τ {(s,s )} {(s,s ) : s,v S T S, v V S, v ˆP : (v,v ) R} 10: end for 11: FS := {v V (v 0,v) R}\LF expore the chidren of v 0 that are not eaves. 12: whie FS /0 do FS contains the frontier vertices to be expored. 13: LFS := {v FS v V : (v,v ) R : s,v S T S v V S} 14: PLFS := {P i I LFS I /0,P i /0, v a,v b P i, v c P j i,v a v b v a v c } 15: for each ˆP PLFS do 16: add new state s to S and a tupe s, ˆP to T S 17: τ := ( s: s,v S T S: v ˆP, v V S,(v,v ) R (s,s)) τ 18: if v ˆP s,v S : v V S (v,v) R then 19: τ := τ {(s,s )} 20: end if 21: end for 22: for each ˆv FS\LFS do 23: add new state s to S and a tupe s,{ ˆv} to TS 24: τ := ( s: s,v S T S: v V S,( ˆv,v ) R(s,s)) τ 25: if s,v S : v V S (v, ˆv) R then 26: τ := τ {(s,s )} 27: end if 28: end for 29: FS := {v V v FS,( v,v) R}\LF 30: end whie 31: define func [.] as [ s](m c ) := µ c (v s ), c {1,,o}, v s V S : s,v S T S 32: define func L as L(s,t) := (t = s)?d : ṽ V S, v V T : s, V S, t, V T T S,(ṽ, v) R LQ(ṽ, v) 33: return S,s ι,τ,σ,[.],l v a v b µ c (v a ) = µ c (v b ), c,1 c o We use the mean of the concentration of the observabe species as a spatia abstraction (superposition) of the observations in a particuar region of the system, avoiding to (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

5 5 enumerate the observations of a ocations. This approach is inspired by [20], [29], where the authors aim to combat the state-exposion probem that woud stem otherwise. Proposition 1. Given a vertex v V of a quad-tree Q = (V,R) and its four chidren v NE,v NW,v SE,v SW the foowing property hods: µ c (v) = µ c(v NE ) + µ c (v NW ) + µ c (v SE ) + µ c (v SW ) 4 Proof: The proof can be easiy derived by expanding the terms of Definition 2. Proposition 2. The number of vertices needed for the quadtree representation Q = (V,R) of a matrix A k,k is upper bounded by k i=0 22i. Proof: The proof foows from the fact that the worst case scenario is when a the eements have different vaues. In this case the cardinaity of the set V is equa to the cardinaity of a fu and compete quaternary tree. For exampe, to represent the matrix A 3,3, it woud require a max number of vertices V = 85. $ $ SE SE v 0 v 1 v 2 v 3 v 4 v 5 v 6 v 7 v 8 v 9 v 10 v 11 v 12 v 13 v 14 v 15 v 16 v 17 v 18 v 19 v 20 SE SE NE,NW,SE,SW (c)$ B =1/2 W =1/2$ s ι $ B =1/2 W =1/2$ s 1 $ NE,SW NW,NE,SE,SW B = 1 W = 0$ s 2 $ B = 0 W = 1$ s 3 $ NW,NE,SE,SW Fig. 3. A checkerboard pattern as a matrix of pixes, the quad-tree representation and the derived quad transition system (c), where B and W denote back and white, respectivey. B. Quad Transition System We now introduce the notion of quad transition system that extends the cassica quad-tree structure, aowing for a more compact exporation for mode checking. Definition 4. A Quad Transition System (QTS) is a tupe Q T S = (S,s ι,τ,σ,[.],l), where: 1) S is a finite set of states with s ι S the initia state; 2) τ S S is the transition reation. We require τ to be non-bocking and bounded-branching: s S, t S : (s,t) τ; s S, if T (s) = {t : (s,t) τ} is the set of a successors of s, then the cardinaity T (s) 4; 3) Σ is a finite set of variabes; 4) [.] is a function [.] : S (Σ [0,b]) that assigns to each state s S and a variabe m Σ a rationa vaue [s](m) in [0,b] with b R + ; 5) L is a abeing function for the transition L : τ 2 D with D = {NW,NE,SE,SW} and with the property that (s,t),(s,t ) τ, with t t it hods that L(s,t) L(s,t ) = /0, t S:(s,t) τ L(s,t) = D. The BUILDINGQUADTRANSITIONSYSTEM agorithm shows how to generate a QTS starting from a quad-tree representation Q=(V,R) of a matrix A k,k and a abeing function LQ : R D. After an initiaization phase (ine 1-4) the agorithm starts to partition the set of equivaent eaves (ine 5-6). Then for each eement in the partition it creates a QTS state with a sef-oop transition (ine 7-10) and a transition from the initia state if the eement represents a root s chid node in the quad-tree. Then it expores a non-eaf quad-tree nodes in a breadth-first fashion and adds new states and transitions to QTS accordingy (ine 12-30). Equivaent quad-tree nodes are represented ony by a singe state in the QTS. The resuting QTS is more compact than the initia quad-tree. Proposition 3. The transition reation of the quad transition system (QTS) Q T S = (S,s ι,τ,σ,[.],l) generated by the BUILD- INGQUADTRANSITIONSYSTEM agorithm has aways a east fixed point, that is s S : T (s) = {s}. Proof: This property hods because the agorithm BUILD- INGQUADTRANSITIONSYSTEM generates one (if the quadtree has ony one vertex) or more (if the quad-tree has mutipe eaves) states with ony a sef-oop transition. Definition 5 (Labeed paths). Given a set B of abes representing the spatia directions, a abeed path (path) of a QTS Q is an infinite sequence π B = s 0 s 1 s 2 of states such that (s i,s i+1 ) τ L(s i,s i+1 ) B /0, i N. Given a state s, we denote LPaths B (s) the set of a abeed paths starting in s, and with π B i the i-th eement of a path π B LPaths B (s). For exampe, in Figure 3, LPaths {NW,SE} (s ι ) = {s ι s 1 s 2 s 2 }. C. TSSL Syntax and Semantics Definition 6 (TSSL syntax). The syntax of TSSL is defined as foows: ϕ ::= m d ϕ ϕ 1 ϕ 2 B ϕ B ϕ B ϕ 1 U k ϕ 2 B ϕ 1 U k ϕ 2 with {, }, d [0,b], b R +, k N >0, B D : B /0, and m Σ, with Σ the set of variabes. From this basic syntax one can derive other two tempora operators: the exist eventuay operator B F k, the fora eventuay operator B F k, the exist gobay operator B G k, and the fora gobay operator B G k defined such that: B F k ϕ := B U k ϕ B F k ϕ := B U k ϕ B G k ϕ := B F k ϕ. B G k ϕ := B F k ϕ. TSSL resembes the cassic CTL ogic [30], with the main difference that the next and unti are not tempora, but spatia operators meaning a change of resoution (or zoom in). The set B seects the spatia directions in which the operator is aowed to work and the parameter k imits the unti (ike bounded unti in bounded mode checking [31]) to operate on a finite sequence of states. In the foowing we provide the TSSL quaitative semantics that, given a spatia mode and a formua representing the pattern to detect, provides a yes/no answer. Definition 7 (TSSL Quaitative Semantics). Let Q = (S,s ι,τ,σ,[.],l) be a QTS, Then, Q satisfies a TSSL formua ϕ, written Q = ϕ, if and ony if Q,s ι = ϕ, where:

6 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 6 Q,s = Q,s = m d Q,s = ϕ Q,s = ϕ 1 ϕ 2 Q,s = B ϕ Q,s = B ϕ Q,s = B ϕ 1 U k ϕ 2 Q,s = B ϕ 1 U k ϕ 2 and Q,s = [s](m) d Q,s = ϕ Q,s = ϕ 1 Q,s = ϕ 2 s : ((s,s ) τ L(s,s ) B /0),Q,s = ϕ s : ((s,s ) τ L(s,s ) B /0),Q,s = ϕ π B LPaths B (s) : i,0 < i k : (Q,π B i = ϕ 2 ) ( j < i,(q,π j = ϕ 1 )) π B LPaths B (s) : i,0 < i k : (Q,π B i = ϕ 2 ) ( j < i,(q,π j = ϕ 1 )) Exampe 2. Checkerboard pattern. The checkerboard pattern from Fig. 3 a) can be characterized with the foowing TSSL formua (B = {SW,NE,NW,SE}): B (( {SW,NE} (m 1)) ( {NW,SE} (m 0))). The eventuay operator can be used to define a the possibe checkerboards of different sizes ess or equa than 4 2 as foows: B F 1 (( {SW,NE} (m 1)) ( {NW,SE} (m 0))) The quaitative semantics is usefu to check if a given spatia mode vioates or satisfies a pattern expressed in TSSL. However, it does not provide any information about how much the property is vioated or satisfied. This information may be usefu to guide a simuation-based parameter exporation for pattern generation. For this reason we equip our ogic aso with a quantitative vauation that provides a measure of satisfiabiity in the same spirit of [17]. Since the vauation of a TSSL formua with spatia operators requires to traverse and to compare regions of space at different resoution, we appy a discount factor of 4 1 on the resut each time a transition is taken in QTS. We choose this vaue to refect that each node represents a partition of the space that is 1 4 smaer than its predecessor. In the foowing, we provide the definition of the TSSL quantitative semantics necessary to measure the satisfaction of a TSSL specification over a given QTS. We show that the sign of this measure indicates either the fufiment (positive sign) or the vioation (negative sign) of a given specification. We then provide a notion of distance between QTSs showing the reation between this distance and the TSSL quaitative and quantitative semantics. Definition 8 (TSSL Quantitative Semantics). Let Q = (S,s ι,τ,σ,[.],l) be a QTS. The quantitative vauation ϕ : S [ b,b] of a TSSL formua ϕ is defined as foows: (s) = b (s) = b m d (s) = ( is )? ([s](m) d) : (d [s](m)) ϕ (s) = ϕ (s) ϕ 1 ϕ 2 (s) = min( ϕ 1 (s), ϕ 2 (s)) B ϕ (s) = 4 1 max π B LPaths B (s) ϕ (πb 1 ) B ϕ (s) = 1 4 min π B LPaths B (s) ϕ (πb 1 ) B ϕ 1 U k ϕ 2 (s) = sup π B LPaths B (s) {min( 1 4 i ϕ 2 (π B i ), inf{ 1 4 j ϕ 1 (π B j ) j < i}) 0 < i k}} B ϕ 1 U k ϕ 2 (s) = inf π B LPaths B (s) {min( 1 4 i ϕ 2 (π B i ), inf{ 1 4 j ϕ 1 (π B j ) j < i}) 0 < i k}} Theorem 1 (Soundness). Let Q = (S,s ι,τ,σ,[.],l) be a QTS, s S a state of Q, and ϕ a TSSL formua. Then, the foowing properties hod for the two semantics: ϕ (s) > 0 = Q,s = ϕ ϕ (s) < 0 = Q,s = ϕ Proof: The proof can be derived by structura induction on the operationa semantics. Remark 1. Theorem 1 provides the basis of the techniques for the parameter synthesis discussed in the foowing sections. ϕ (s) enabes the process of quantitative vauation of a TSSL formua ϕ over a QTS by performing the recursive computation presented in Definition. 8. The computationa cost is inear in the QTS size and poynomia in the ength of the formua. It is worth to note that, in the case ϕ (s) = 0, it is not possibe to infer whether Q vioates or satisfies a TSSL formua ϕ and ony in this particuar case we need to resort to the quaitative semantics for determining it. We now introduce a notion of distance between two given QTSs. This measure quantifies, by recursivey exporing the corresponding pair of nodes of two QTSs, the max absoute difference between the evauation of the variabes in the pair of nodes discounted by a factor 1/4 k. The term k is the recursion eve of the expored pair of nodes. Higher eve eads to smaer partitions of the space that the pair of nodes represent. Consequenty, their max absoute difference is ess important. Since the nodes correspond to partitions of the space, the max distance computes the overa worst discrepancy between corresponding partitions of the space. Definition 9. (QTS Max Distance) The max distance of two QTSs Q (1) = (S (1),s (1),τ (1),Σ,[.] (1),L (1) ) and Q (2) = (S (2),s (2),τ (2),Σ,[.] (2),L (2) ) is defined as: d (Q (1),Q (2) ) = n (s (1),s (2),0) where n : S S N [0,b] is the max distance between states of different QTSs such that: 1 max 4 k m Σ [s(1) ] (1) (m) [s (2) ] (2) (m) if (s (1),s (1) ) τ (1) (s (2),s (2) ) τ (2) n (s (1),s (2),k) = max n ( s (1), s (2),k + 1) otherwise ( s (1), s (2) ) S S = {( s (1), s (2) ) s (1) S (1), s (2) S (2) L (1) (s (1), s (1) ) L (2) (s (2), s (2) ) /0} It is worth noting that if two pictures are the same, but they have different number of pixes then their QTS representations wi be equivaent and their max difference wi be zero. We now introduce a second theorem, showing the correctness of the quaitative semantics w.r.t. the quantitative semantics. According this theorem if the max distance between two QTSs is ess than the quantitative vauation of a TSSL formua ϕ over the first QTS satisfying ϕ, then we aso know that the other QTS satisfies ϕ. Theorem 2 (Correctness). Given a TSSL formua ϕ and two QTSs Q (1) = (S (1),s (1),τ (1),Σ,[.] (1),L (1) ) and Q (2) = (S (2),s (2),τ (2),Σ,[.] (2),L (2) ) and two states s (1) S (1) and s (2) S (2). If Q (1),s (1) satisfies the formua ϕ and the max distance n (s (1),s (2),0) is ess than the quantitative evauation ϕ (s (1) ) (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

7 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 7 of ϕ over Q (1) then aso Q (2),s (2) satisfies the same formua ϕ. Formay: Q (1),s (1) = ϕ n (s (1),s (2),0) < ϕ (s (1) ) Q (2),s (2) = ϕ Proof: (Sketch) We can distinguish the foowing cases: case ϕ := : in this case the theorem is true foowing the definition of the quaitative semantics (see Definition 7) for which both Q (1),s (2) and Q (2),s (2) satisfy. case ϕ := m d: In this case we have that: n (s (1),s (2),0) < ϕ = m d (s (1) ) (see hypothesis) m d (s (1) ) = [s (1) ] (1) (m) d (Def. 8) (c) [s (1) ] (1) (m) n (s (1),s (2),0) d > 0 (from and ) (d) [s (1) ] (1) (m) [s (2) ] (2) (m) n (s (1),s (2),0) (Def. 9) If we substitute n (s (1),s (2),0) with [s (1) ] (1) (m) [s (2) ] (2) (m) in (c), given (d), we can safey obtain: (e) [s (1) ] (1) (m) [s (1) ] (1) (m) [s (2) ] (2) (m) d > 0 Using the property of the absoute difference, we have: (f) [s (2) ] (2) (m) [s (1) ] (1) (m) [s (1) ] (1) (m) [s (2) ] (2) (m) If we substitute [s (2) ] (2) (m) with [s (1) ] (1) (m) [s (1) ] (1) (m) [s (2) ] (2) (m) in (e), given (f) we have [s (2) ] (2) (m) d > 0. Finay, using the theorem 1 we obtain the foowing: [s (2) ] (2) (m) d > 0 m d (s (2) ) > 0 Q (2),s = ϕ case ϕ := m d: In this case we have that: (g) n (s (1),s (2),0) < ϕ = m d (s (1) ) (see hypothesis) (h) m d (s (1) ) = d [s (1) ] (1) (m) (Def. 8) (i) d [s (1) ] (1) (m) n (s (1),s (2),0) > 0 (from (g) and (h)) (j) [s (1) ] (1) (m) [s (2) ] (2) (m) n (s (1),s (2),0) (Def. 9) If we substitute n (s (1),s (2),0) with [s (1) ] (1) (m) [s (2) ] (2) (m) in (i), given (j), we can safey obtain: (k) d [s (1) ] (1) (m) [s (1) ] (1) (m) [s (2) ] (2) (m) > 0 Using the property of the absoute difference, we have: () [s (2) ] (2) (m) [s (1) ] (1) (m) [s (1) ] (1) (m) [s (2) ] (2) (m) If we substitute [s (2) ] (2) (m) with [s (1) ] (1) (m) [s (1) ] (1) (m) [s (2) ] (2) (m) in (k), given () we have d [s (2) ] (2) (m) > 0. Finay, using Theorem 1 we obtain the foowing: d [s (2) ] (2) (m) > 0 m d (s (2) ) > 0 Q (2),s = ϕ a the other cases: If Q (1),s (1) = ϕ then we have: 1 (1): ϕ (s (1) 4 j b: j N ) = 1 (2): 4 j ([s (1) ] (1) (m (1) ) d) : j N Situation (1) may occur when one of the subformuae of ϕ is and the proof is equivaent to the case of ϕ :=. Situation (2) can be proved in a simiar way as the case ϕ := m d. Proposition 4. Given a TSSL formua ϕ and two QTSs Q (1) = (S (1),s (1),τ (1),Σ,[.] (1),L (1) ) and Q (2) = (S (2),s (2),τ (2),Σ,[.] (2),L (2) ) then: Q (1) = ϕ d (Q (1),Q (2) ) < ϕ (s (1) ) Q (2) = ϕ Proof: This is a specia case of Theorem 2 where s (1) and s (2) are the initia states s (1), s (2) of Q (1) and Q (2) respectivey. Remark 2. The correctness theorem impies that the higher the quantitative vauation of a TSSL formua is with respect to a QTS, the harder it is to vioate the formua by perturbing the QTS since the maximum distance between the perturbation and the origina QTS must be at east equa to the quantitative vauation. In other words, a higher positive quantitative vauation means a more robust satisfaction of a formua under QTS perturbations. That is why the quantitative vauation of a TSSL formua is aso caed its robustness degree. V. TSSL PATTERN CLASSIFIERS A QTS can be seen in the context of muti-resoution representation, since the nodes that appear at deeper eves provide information for higher resoutions. Therefore, a TSSL formua can effectivey capture properties of an image. However, it is difficut to write a formua that describes a desired property, such as a pattern. Here, we propose to use machine-earning techniques to find such a formua from given sets of positive (Y + ) and negative (Y ) exampes. We first define a abeed data set from the given data sets Y + and Y as L = {(Q y,+) y Y + } {(Q y, ) y Y }, where Q y is the QTS generated from y. Then, we separate the data set L into disjoint training and testing sets L L,L T. In machine-earning, the training set is used to earn a cassifier for a target cass, e.g. +, and the testing set is used to measure the accuracy of the cassifier. We empoy RIPPER [32], a rue based earner, to earn a cassifier from L L, and then transate the cassifier into a TSSL formua characterizing +. The cassifier is composed of a set of rues. Each rue is described as r i : C i i, where C i is a booean formua over inear predicates over the variabes of the states of a QTS, e.g. [s](m) > d, and i takes vaues from the abe set {+, }. A inear predicate for a state s S can be written as a TSSL formua via the QTS path from the root s ι to s as a state s is uniquey represented using the existentia ( ) and next ( ) operators aong the path from s ι to s. Therefore, each C i can be transated into an equivaent TSSL formua Φ i. The cassification rues are interpreted as (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

8 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 8 nested if-ese statements. Hence, an equivaent TSSL formua for the desired property is defined as foows: ( Φ + := Φ j Φ i ), (4) j R + i=1,..., j 1 where R + is the set of indices of rues r i with i = +, and Φ i is the TSSL formua obtained from C i. Fig. 4. Sampe sets of images from the sets Y (1) + and Y (1) for the LS pattern. Exampe 3. LS pattern. For the LS pattern from Exampe 1, we generate a data set Y (1) + containing 8000 positive exampes by simuating the reaction-diffusion system (3) from random initia conditions with parameters R and D 1. Simiary, to generate the data set Y (1) containing 8000 negative exampes, we simuate system (3) from random initia conditions. However, in this case we use R and randomy choose the diffusion coefficients from R 2 [0,30]. As stated before, we ony consider the observation of a system in steady-state, for this reason, simuated trajectories that do not reach steady state-in 60 time units are discarded. A sampe set of images from the sets Y (1) + and Y (1) is shown in Figure 4. We generate a abeed set L (1) of QTS from these sets, and separate L (1) into L (1),L(1). We use the RIPPER agorithm impemented in Weka [33] to earn a cassifier from L (1) L. The earning step took 228.5sec on an imac with a Inte Core i5 processor at 2.8GHz with 8GB of memory. The cassifier consists of 24 rues. The first rue is r 1 :(R 0.59) (R 0.70) (R.NW.NW.NW.SE 0.75) (R.NW.NW.NW.NW 0.45) +, R denotes the root of a QTS, the abes of the chidren are shown in Figure 2, and + indicates the presence of the pattern. Rue r 1 transates to the foowing TSSL formua: Φ 1 :(m 0.59) (m 0.70) ( NW NW NW SE m 0.75) ( NW NW NW NW m 0.45). (5) We define the TSSL formua Φ (1) + characterizing the pattern as in (4), and mode check QTSs from L (1) T ( L (1) T = 8000) against Φ (1) +, which yieds a high prediction accuracy (96.11%) with 311 miss-cassified QTSs. FP and SS patterns. We foow the above steps to generate data sets Y (i) +,Y (i), generate abeed data sets L (i) L,L(i) T, and finay earn formuas Φ (i) + for the FP and SS patterns corresponding to diffusion coefficient vectors D i, i = 2,3 from Exampe 1. The mode checking of the QTSs from the corresponding test sets yieds high prediction accuracies 98.01%, and 93.13% for Φ (2) +, and Φ (3) +, respectivey. L T VI. PARAMETER SYNTHESIS FOR PATTERN GENERATION In this section we present the soution to Probem 1, i.e. a framework to synthesize parameters p P of a reactiondiffusion network S (1) such that the observations of system S (p) satisfy a given TSSL formua Φ. First, we show that the parameters of a reaction-diffusion system that produce trajectories satisfying the TSSL formua can be found by optimizing quantitative mode checking resuts. Second, we incude the optimization in a supervised iterative procedure for parameter synthesis. We sighty abuse the terminoogy and say that a trajectory x(t),t 0 of system S (p) satisfies Φ if the QTS Q = (S,s ι,τ,σ,[.],l) of the corresponding observation, H(x( t)), satisfies Φ, i.e Q = Φ, or Φ (s ι ) > 0. We define an induced quantitative vauation of a system S (p) and a set of initia conditions X 0 from a TSSL formua Φ as: Φ (S (p) ) = min x 0 X 0 { Φ (s ι ) Q = (S,s ι,τ,σ,[.],l) (6) is QTS of H(x( t)),x(0) = x 0 } The definition of the induced vauation of a system S (p) impies that a trajectories of S (p) originating from X 0 satisfy Φ if Φ (S (p) ) > 0. Therefore, it is sufficient to find p that maximizes (6). It is assumed that the ranges P = P 1... P P of the design parameters are known. Therefore, the parameters maximizing (6) can be found with a greedy search on a quantization of P. However, the computation of Φ (S (p) ) for a given p P is expensive, since it requires to perform the foowing steps for each x 0 X 0 : simuating the system S (p) from x 0, generating QTS Q of the corresponding observation, and quantitative mode checking of Q against Φ. Here, we use the partice swarm optimization (PSO) agorithm [34] over P with (6) as the fitness function. The choice of PSO is motivated by its inherent distributed nature, and its abiity to operate on irreguar search spaces. In particuar, PSO does not require a differentiabe fitness function. Exampe 4. LS pattern. We consider the reaction-diffusion network from Exampe 1 and the TSSL formua Φ (1) + corresponding to the LS pattern from Exampe 3. We assume that the parameters of the oca dynamics are known, R = [1, 12, 1,16], and the diffusion coefficients D 1 and D 2 are set as the design parameters with P = R 2 [0,30]. We impement PSO to find p P maximizing the induced vauation (6). The PSO computation was distributed on 16 processors at 2.1GHz on a custer, and the running time was around 18 minutes. The optimized parameters are D 1 = 2.25 and D 2 = 29.42, and the vauation of the system is A set of observations obtained by simuating S ([2.25,29.42]) is shown in Figure 6-. Note that, whie a the observations have some spatia periodicity indicating the presence of a pattern, they are sti different from the desired LS pattern. FP and SS patterns. We aso appy the PSO agorithm on the same setting expained above to maximize the induced vauation (6) for the TSSL formuas Φ (2) + (FP pattern) and Φ (3) + (SS pattern) from Exampe 3. The optimized parameters are [0.083,11.58] and [1.75,7.75] for Φ (2) + and Φ (3) +, respec (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

9 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 9 tivey. Sets of observations obtained by simuating systems S ([0.083,11.58]) and S ([1.75,7.75]) are shown in Figure 5. In contrast with the LS pattern, the observations are simiar to the ones from the corresponding data sets i.e. Y (2) + and Y (3) +. Fig. 5. Sampe set of observations obtained by simuating S ([0.083,11.58]) and S ([1.75,7.75]). (c) Fig. 6. Sampe set of observations obtained by simuating S ([2.25,29.42]), S ([3.75,28.75]), and (c) S ([6.25,29.42]). Remark 3. In this paper, we consider the observations generated from a given set of initia conditions X 0. However, the initia condition can be set as a design parameter and optimized in PSO over a given domain R K K N [a,b]. As seen in Exampe 4, it is possibe that simuations of the system corresponding to optimized parameters do not necessariy ead to desired patterns. This shoud not be unexpected, as the formua refects the origina training set of positive and negative exampes, and was not aware that these new simuations do not produce good patterns. A natura extension of our method shoud aow to add the newy obtained simuations to the negative training set, and to reiterate the whoe procedure. This approach is summarized in the INTERACTIVEDESIGN agorithm. Agorithm INTERACTIVEDESIGN Input: Parametric reaction-diffusion system S, ranges of parameters P, a set of initia states X 0, sets of observations Y + and Y Output: Optimized parameters p, the corresponding vauation γ (no soution if γ < 0) 1: whie True do 2: Φ = Learning(Y +,Y ) 3: {p,γ} = Optimization(S,X 0,Φ) γ is the induced vauation of S (p) 4: if γ < 0 then return p,γ 5: end if 6: UserQuery: Show observations of trajectories of S (p) originating from X 0. 7: if User approves then return p, γ 8: ese 9: Y = Y {H(x( t)) x(t),t 0, 10: end if 11: end whie is generated by S (p),x(0) X 0 }. We start with the user defined sets of observations Y + and Y, and earn a TSSL formua Φ from the QTS representations of the observations (Section V). Then, in the optimization step, we find a set of parameters p that maximizes γ = Φ (S (p) ). If γ < 0, then we terminate the agorithm as parameters producing observations simiar to the ones from the set Y + with respect to the TSSL formua Φ coud not be found. If γ 0, then the observations of system S (p) satisfy Φ. Finay, the user inspects the observations generated from the reaction-diffusion system with the optimized set of parameters S (p). If the observations are simiar to the ones from the set Y +, then we find a soution. If, however, the user decides that the observations do not contain the pattern, then we add observations obtained from system S (p) to Y, and repeat the process, i.e earn a new formua, run the optimization unti the user terminates the process or the optimization step fais (γ < 0). Exampe 5. LS pattern. We appy INTERACTIVEDESIGN agorithm to the system from Exampe 4. A sampe set of observations obtained in the first iteration is shown in Figure 6-. We decide that these observations are not simiar to the ones from the set Y (1) + shown in Figure 4-, and add these 250 observations generated with the optimized parameters to Y (1) (ine 9). In the second iteration, the optimized parameters are D 1 = 3.75 and D 2 = 28.75, and the observations obtained by simuating S ([3.75,28.75]) are shown in Figure 6-. We continue by adding these to Y (1). The parameters computed in the third iteration are D 1 = 6.25 and D 2 = The observations obtained by simuating S ([6.25,29.42]) are shown in Figure 6-(c). Athough the optimized parameters are different from D 1, which was used to generate Y (1) +, the observations of S ([6.25,29.42]) are simiar to the ones from the set Y (1) + and we terminate the agorithm. Remark 4. As mentioned earier, the mode checking procedure (computation of quantitative vauation) is very efficient (inear in the size of the system and poynomia in the ength of the formua). For instance, computing the quantitative vauation for the TSSL formua corresponding to the LS pattern against a 32 by 32 image takes about 0:5 seconds on an imac with a Inte Core i5 processor at 2.8GHz with 8GB of memory. However, a arge number of unknown parameters woud be probematic for the deveoped parameter synthesis framework since such a system requires a very arge swarm popuation and a high number of iterations for PSO independent of the fitness function (e.g. quantitative mode checking). VII. COMPARISON: TSSL AND LINEAR CLASSIFIERS In this section, we provide a comparison between TSSL and we-known earning agorithms based on inear cassifiers. A. Linear Cassifiers Assume we have m data points (x i,y i ) : i = 1,2,...,m, where x i R d is a vector containing d features that correspond to the ith exampe in the training set and y i { 1,+1} is the cass abe associated with x i. A inear cassifier is a function of the form h(x) = sgn(w T x +b), where w R d is the norma vector corresponding to the hyperpane {x R d : w T x+b = 0} and b R is the hyperpane s bias, and sgn is the signum (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

10 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 10 indicator function. The Eucidean distance between a point in the feature space and the hyperpane is caed the geometric margin, γ(x) = wt x+b w. Notice that the geometric margin of a point can be viewed as a distance to pattern satisfaction, since the cass prediction of a testing point with a higher geometric margin is stronger. The goa of the earning probem is to find w and b such that h(x) correcty cassifies the training data points. Severa agorithms have been proposed in the machine earning iterature to earn a cassifying hyperpane for a given data set. In this paper, we use three such agorithms and compare the resuts with TSSL: 1) Support Vector Machines (SVM) [3]: A hyperpane is chosen such that the minimum margin among a the data points in the training set is maximized. It is shown in [3] that a SVM can be earned when the data is not ineary separabe by soving a quadratic programming probem (Soft Margin Method). Furthermore, SVM can be kerneized, i.e., kerne functions can be used to map the origina data points to a higher dimensiona space where the data is ineary separabe, which resuts in a noninear cassifier in the origina feature space. 2) Fisher Linear Discriminant (FLD) [3]: A hyperpane is obtained by maximizing the between-cass variance whie minimizing the within-cass variances. 3) Kozinec s Agorithm [22]: A separating hyperpane is earnt in an iterative procedure that appies corrections to cassify each point in the training set. B. Cassification Rate In this section, we compare the effectiveness of TSSL cassifiers and the inear cassifiers described above. We created three distinct training sets for the LS, SS, and FP Turing patterns using the procedure discussed in Section V. Each set consists of 4000 positive and 4000 negative exampes other images were generated to test the resuts. Cassifier Correct cassification rates LS FP SS TSSL(RIPPER) 96.7 % 96.1 % 95.6 % SVM hyperpane 94.5 % 91.7 % 95.3 % FLD 96.5 % 93.9 % 92.8 % Kozinec s hyperpane 95.2 % 89.1 % 92.4 % TABLE I CLASSIFICATION RATES OF TSSL (LEARNED BY RIPPER) COMPARED TO LINEAR CLASSIFIERS (THE CLASSIFICATION RATES ARE COMPUTED FOR A TESTING SET CONSISTING OF 8000 EXAMPLES) Remark 5. Learning a SVM requires determining particuar design parameters (e.g., proper kerne functions and their parameters, the so-caed parameter C in the soft margin method, proper features). These parameters need to be finetuned using techniques such as cross-vaidation in order to earn an effective cassifier. This is a difficut and timeconsuming process for a arge number of data points and features. On the other hand, TSSL works effectivey without the need of tuning any parameters as shown in Tabe I. C. Distance to Pattern A very important feature of TSSL is its quantitative semantics, which can be used as a measure of distance to satisfaction. One can use this measure to compare two patterns and determine which one is a better. Furthermore, this metric is used as fitness function in partice swarm optimization (Section VI) to synthesize system parameters. It is interesting to note that, for inear cassifiers as described above, one can aso view the (e.g., Eucidean) distance between a data point and the cassifier hyperpane (geometrica margin) as distance to satisfaction. In this section, we hypothesize that the distance given by TSSL is more meaningfu and more usefu for optimization based pattern synthesis than that given by inear cassifiers. TSSL quantitative vauation x Time (s) SVM geometric margin time (s) (c) Fig. 7. Exampes of images used as training set to compare TSSL with SVM. Positive Exampes (SS). Negative Exampes We considered two types of features for the inear cassifiers. First, we simpy considered the normaized concentrations of species 1 in each ce of the grid (i.e., the feature vector is dimensiona for our grid). Second, we used histograms of oriented gradients, which were created according to the methodoogy presented in [11]. For each type of inear cassifier, and for each type of feature, we earnt the cassifier, tested it against the testing set, and kept the one with the best cass-action rate to compare it with TSSL. Tabe I shows the resuts of this comparison. Fig. 8. Formation of LS in steady state TSSL quantitative vauation with respect to Φ (1) + (see Eqn. (5)) at various time steps (c) SVM geometric margin Figures 8, 9, and 10 show a comparison between the TSSL and SVM metrics. Each figure shows the evoution of the metric over time for each of the three considered patterns LS, FP, and SS. In a three cases, the TSSL quantitative vauation is better behaved. Indeed, the TSSL curves have fewer oca optima (e.g., Fig. 9 and 10), and reach goba maxima at steady state (e.g., Fig. 8 and 9). VIII. HIGHER-ORDER STATISTICS IN TSSL In previous sections, TSSL formuae have been earned using the first moments as features in nodes of the quad-trees (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

11 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 11 TSSL quantitative vauation Time (s) SVM geometric margin time (s) Fig. 9. Formation of FP in steady state TSSL quantitative vauation with respect to Φ (2) + at various time steps (c) SVM geometric margin TSSL Quantitative Vauation 10 x Time (c) Time Fig. 10. Formation of SS in steady state TSSL quantitative vauation with respect to Φ 3 + at various time steps (c) SVM geometric margin In other words, we have assumed that Σ = {m} in the definition of quad transition systems where m denotes mean vaues. In this section, we study the effect of adding higher moments to the set of variabes Σ. In particuar, we added variance of concentrations of specie 1 to the set of variabes in the QTS and investigated how it improves the resuts. Assume that the set of variabes in the QTS is Σ = {m,v} where m represents mean vaues and v represents variance, respectivey. We repeated the procedure presented in Sections V and VI and observed that: SVM geometric margin 1) Improvement in pattern recognition: Adding variance significanty reduces the ength and number of RIPPER cassification rues and thus the TSSL formua that represents a given pattern wi be much shorter. Athough the enhancement in prediction accuracy is imited and often negigibe, this has a notabe effect on the computation time, since shorter cassification rues are easier and faster to earn. 2) Improvement in pattern synthesis: The compexity of TSSL quantitative vauation for a given formua is proportiona to the ength of the formua. Therefore, a shorter TSSL formua resut in a faster computation of Eqn. (6). Consequenty, the optimization step in Agorithm INTERACTIVEDESIGN is performed faster since we need to compute the quantitative vauation at every iteration of PSO. Exampe 6. LS pattern. We considered the experiment described in Exampe 3 and repeated the same procedure (Same training and testing sets and simuation variabes) using both first and second order statistics for the LS pattern. The (c) earning step took 23.3sec on the same computer described in Exampe 3. The cassifier consists of 8 rues. Note that the experiment of exampe 3 consisted of 24 rues which were earned in 228sec. The cassifier that is buit using mean and variance of observed concentrations yieds a high prediction accuracy (98.27%). IX. CONCLUSION AND FUTURE WORK We defined a tree spatia superposition ogic (TSSL) whose semantics is naturay interpreted over quad trees of partitioned images. We showed that formuas in this ogic can be efficienty earned from positive and negative exampes. We defined a quantitative semantics for TSSL and combined with an optimization agorithm to deveop a supervised, iterative procedure for synthesis of pattern-producing parameters in a network of ocay interacting dynamica systems. Whie the experiments show that the current version of the ogic works quite we and can accommodate transationa and rotationa symmetries commony found in bioogy patterns, there are severa directions of future work. First, we do not expoit the fu semantics of the ogic in this paper. In future work, we pan to investigate reasoning about mutipe branches and using the unti operator. Second, we pan to appy this method to more reaistic networks, such as popuations of ocay interacting engineered ces. We expect that experimenta techniques from synthetic bioogy can be used to tune existing synthetic gene circuits to produce goba desired patterns. REFERENCES [1] J. Goden and H. Yoon, Heterocyst formation in anabaena, Curr Opin Microbio., vo. 1, no. 6, pp , [2] R. Scherrer and V. Shu, Structure, partia eementa composition, and size of thiopedia rosea ces and pateets, Can J Microbio., vo. 32, no. 7, pp , [3] C. M. Bishop, Pattern Recognition and Machine Learning. Springer, [4] A. Jain, R. Duin, and J. Mao, Statistica pattern recognition: A review, IEEE Transactions on pattern anaysis and machine earning, vo. 22, pp. 4 37, [5] T. Pavidis, Structura Pattern Recognition. Springer-Verag, [6] R. C. Vetkamp and M. Hagedoorn, State-of-the-art in shape matching, Principes of Visua Information Retrieva, Tech. Rep., [7] E. A. Emerson, Tempora and moda ogic, in Handbook of Theoretica Computer Science: Forma Modes and Semantics, J. van Leeuwen, Ed. North-Hoand Pub. Co./MIT Press, 1990, vo. B, pp [8] E. M. M. Carke, D. Peed, and O. Grumberg, Mode checking. MIT Press, [9] A. M. Turing, The chemica basis of morphogenesis, Phiosophica Transactions of the Roya Society of London, vo. 327, pp , [10] B. Juesz, Textons, the eements of texture perception, and their interactions, Nature, vo. 290, pp , [11] N. Daa and B. Triggs, Histograms of oriented gradients for human detection, in Proc. of CVPR 2005: the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vo. 1, June 2005, pp [12] S. Beongie, J. Maik, and J. Puzicha, Shape Matching and Object Recognition Using Shape Contexts, IEEE Transactions on Pattern Anaysis and Machine Inteigence, vo. 24, pp , [13] D. G. Lowe, Object recognition from oca scae-invariant features, in Proc. of the Internationa Conference on Computer Vision, vo. 2, 1999, pp [14] S. J. Russe and P. Norvig, Artificia Inteigence: A Modern Approach. Prentice Ha, (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.

12 This artice has been accepted for pubication in a future issue of this journa, but has not been fuy edited. Content may change prior to fina pubication. Citation information: DOI /TCNS , IEEE Transactions on Contro of Network Systems 12 [15] A. Rizk and F. Fages, From Mode-Checking to Tempora Logic Constraint Soving, in Proc. of CP 2009: the 15th Internationa Conference on Principes and Practice of Constraint Programming, Lisbon, Portuga, September, ser. Lecture Notes in Computer Science, vo Springer, 2009, pp [16] A. Donzé and O. Maer, Robust satisfaction of tempora ogic over rea-vaued signas, in Proc. of FORMATS 2010, the 8th Internationa Conference on Forma Modeing and Anaysis of Timed Systems, Kosterneuburg, Austria, September 8 10, vo. 6246, 2010, pp [17] A. Donzé, E. Fanchon, L. M. Gattepaie, O. Maer, and P. Tracqui, Robustness anaysis and behavior discrimination in enzymatic reaction networks, PLoS One, vo. 6, no. 9, p. e24246, [18] G. Fainekos and G. Pappas, Robust Samping for MITL specifications, in Proc. of FORMATS 2007, the 5th Internationa Conference on Forma Modeing and Anaysis of Timed Systems, ser. Lecture Notes in Computer Science, 2007, vo. 8044, pp [19] G. E. Fainekos and G. J. Pappas, Robustness of tempora ogic specifications for continuous-time signas, Theor. Comput. Sci., vo. 410, no. 42, pp , [20] R. Grosu, S. Smoka, F. Corradini, A. Wasiewska, E. Entcheva, and E. Bartocci, Learning and detecting emergent behavior in networks of cardiac myocytes, Communications of the ACM, vo. 52, no. 3, pp , [21] L. de Afaro, M. Faea, and M. Stoeing, Linear and branching system metrics, IEEE Transactions on Software Engineering, vo. 35, no. 2, pp , [22] M. I. Schesinger and V. Havac, Ten ectures on statistica and structura pattern recognition. Springer, 2002, vo. 24. [23] W. Ren and R. Beard, Distributed Consensus in Muti-vehice Cooperative Contro: Theory and Appications. Springer-Verag, London, [24] M. Mesbahi and M. Egerstedt, Graph Theoretic Methods for Mutiagent Networks. Princeton University Press, Princeton, NJ, [25] F. Buo, J. Corts, and S. Martnez, Distributed Contro of Robotic Networks. A Mathematica Approach to Motion Coordination Agorithms. Princeton University Press, [26] E. A. Go, E. Bartocci, and C. Beta, A forma methods approach to pattern synthesis in reaction diffusion systems, in 53rd IEEE Conference on Decision and Contro (CDC), Los Angees, CA, 2014, pp [27] R. Coantes, Agorithm aey. Dr. Dobb s journa, December [28] R. Finke and J. Bentey, Quad trees a data structure for retrieva on composite keys, Acta Informatica, vo. 4, no. 1, pp. 1 9, [29] Y. Kwon and G. Agha, Scaabe modeing and performance evauation of wireess sensor networks, in Rea-Time and Embedded Technoogy and Appications Symposium, Proceedings of the 12th IEEE, 2006, pp [30] E. M. Carke and E. A. Emerson, Design and synthesis of synchronization skeetons using branching-time tempora ogic, in Proc. of Logic of Programs Workshop, ser. Lecture Notes in Computer Science, vo. 131, 1982, pp [31] A. Biere, A. Cimatti, E. M. Carke, O. Strichman, and Y. Zhu, Bounded mode checking, Advances in computers, vo. 58, pp , [32] W. W. Cohen, Fast effective rue induction, in Proceedings of the Twefth Internationa Conference on Machine Learning. Morgan Kaufmann, 1995, pp [33] M. Ha, E. Frank, G. Homes, B. Pfahringer, P. Reutemann, and I. H. Witten, The weka data mining software: An update, SIGKDD Expor. News., vo. 11, no. 1, pp , Nov [34] J. Kennedy and R. Eberhart, Partice swarm optimization, in Proceedings of the IEEE Internationa Conference on Neura Networks, vo. 4, 1995, pp Ezio Bartocci received the BS degree in computer science, the MS degree in bioinformatics, and the PhD degree in information sciences and compex systems from the University of Camerino, Itay, in 2002, 2005, and 2009, respectivey. From 2010 to 2012, he was a research associate at Department of Appied Math and Statistics and research scientist at Department of Computer Science at the State University of New York at Stony Brook. He joined the Facuty of Informatics at Vienna University of Technoogy in 2012 as university assistant. Since 2015 is a tenure-track assistant professor at the Department of Computer Engineering at Vienna University of Technoogy. The primary focus of his research is to deveop forma methods, computationa toos and techniques which support the modeing and the automated anaysis of compex computationa systems, incuding software systems, cyber-physica systems and bioogica systems. Ebru Aydin Go (M 12) is an assistant professor at Orta Dogu Teknik Universitesi (ODTU) in the Department of Computer Engineering. She received her B.S. degree in Computer Engineering from ODTU, Ankara, Turkey, in 2008, M.S. degree in Computer Science from Ecoe Poytechnique Federae de Lausanne, Lausanne, Switzerand, in 2010 and Ph.D. degree in Systems Engineering from Boston University, Boston, MA, USA in Prior to joining ODTU in 2016, she worked as a Site Reiabiity Engineer at Googe. Her research interests incude forma verification and contro, probabiistic verification, hybrid systems, software/reease evauation, verification and design of cyber-physica systems. Iman Haghighi (S 16) received the B.S. degree in mechanica engineering from Sharif University of Technoogy, Tehran, Iran in 2013, and M.S. degree in systems engineering from Boston University, Boston, MA, USA in He is currenty working towards the Ph.D. degree in systems engineering at the division of systems engineering, Boston University, Boston, MA, USA. His current research interests incude forma verification and synthesis in networked systems, pattern recognition and formation, and spatio-tempora ogic synthesis. Cain Beta (SM 11) is a Professor in the Department of Mechanica Engineering at Boston University, where he hods the Tegan Famiy Distinguished Facuty Feowship. He is the Director of the BU Robotics Lab, and is aso affiiated with the Department of Eectrica and Computer Engineering, the Division of Systems Engineering at Boston University, the Center for Information and Systems Engineering (CISE), and the Bioinformatics Program. His research focuses on dynamics and contro theory, with particuar emphasis on hybrid and cyber-physica systems, forma synthesis and verification, and appications in robotics and systems bioogy. He received the Air Force Office of Scientific Research Young Investigator Award and the Nationa Science Foundation CAREER Award (c) 2016 IEEE. Persona use is permitted, but repubication/redistribution requires IEEE permission. See for more information.