arxiv: v7 [cs.db] 24 Jul 2015

Transcription

1 Relational oundations or unctorial Data Miration David I. Spivak, Ryan Wisnesky Massachusetts Institute o Technoloy {dspivak, wisnesky}@math.mit.edu arxiv: v7 [cs.db] 24 Jul 2015 July 28, 2015 Abstract We study the data transormation capabilities associated with schemas that are presented by directed multiraphs and path equations. Unlike most approaches which treat raph-based schemas as abbreviations or relational schemas, we treat raph-based schemas as cateories. A schema S is a initely-presented cateory, and the collection o all S-instances orms a cateory, S Inst. A unctor between schemas S and T, which can be enerated rom a visual mappin between raphs, induces three adjoint data miration unctors, Σ : S Inst T Inst, Π : S Inst T Inst, and : T Inst S Inst. We present an alebraic query lanuae QL based on these unctors, prove that QL is closed under composition, prove that QL can be implemented with the select-projectproduct-union relational alebra (SPCU) extended with a key-eneration operation, and prove that SPCU can be implemented with QL. 1 Introduction In this paper we describe how to implement unctorial data miration [16] usin relational alebra, and vice versa. In the unctorial data model [16], a database schema is a initely-presented cateory (a directed multiraph with path equations [4]), and a database instance on a schema S is a unctor rom S to the cateory o sets, Set. The database instances on a schema S constitute a cateory, denoted S Inst, and a unctor between schemas S and T induces three data miration unctors: : T Inst S Inst, deined as (I) := I, and its let and riht adjoints Σ : S Inst T Inst and Π : S Inst T Inst, respectively. We deine a unctorial query lanuae, QL, in which every query denotes a data miration unctor, prove that QL is closed under composition, and deine a translation rom QL to the select-project-product-union (SPCU) relational alebra extended with an operation or creatin lobally unique IDs (GUIDs) 1. In addition, we describe how to implement the SPCU relational alebra with QL. Our primary motivation or this work is to use SQL as a practical deployment platorm or the unctorial data model. Usin the results o this paper we have implemented a SQL compiler or QL, and a (partial) translator rom SQL to QL, as part o a visual schema mappin tool available at cateoricaldata.net/ql.html. We are interested in the unctorial data model because we believe its cateory-theoretic underpinnins provide a better mathematical oundation or inormation interation than the relational model. or example, many practical relational systems (includin SQL-based RDBMSs) are not actually relational, but are based on bas and unique row IDs like the unctorial data model. Numerous advantaes o the unctorial data model are described in [16] and [8]. 1.1 Related Work Althouh cateory presentations directed multiraphs with path equivalence constraints are a common notation or schemas [5], prior work rarely treats such schemas cateorically [8]. Instead, most prior work treats such schemas as abbreviations or relational schemas. or example, in Clio [10], users draw lines connectin related elements between two schemas-asraphs and Clio enerates a relational query (semantically similar to our Σ) that implements the user s intended data transormation. Behind the scenes, the user s correspondence is translated into a ormula in the relational lanuae o second-order tuple eneratin dependencies, rom which a query is enerated [7]. In many ways, our work is an extension and improvement o Rosebruh et al s initial work on understandin cateory presentations as database schemas [8]. In that work, the authors identiy the Σ and Π data miration unctors, but they do not identiy as their adjoint. Moreover, they do not implement Σ and Π usin relational alebra, and they do not ormalize a query lanuae or investiate the behavior o Σ and Π Extended version o the DBPL 2015 paper. Work unded by ONR rants N and N An operation to create IDs is required because unctorial data miration can create constants not contained in input instances. 1

2 with respect to composition; neither do they consider attributes. Our mathematical development diveres rom their subsequent work on sketches [13], in which some or all o, Σ, and Π data miration unctors may no loner exist. Cateory-theoretic techniques were instrumental in the development o the nested relational data model [17]. However, we do not believe that the unctorial data model and the nested relational model are closely connected. The unctorial data model is more closely related to work on cateorical loic and type theory, where operations similar to Σ, Π, and oten appear under the sloan that quantiication is adjoint to substitution [12]. 1.2 Contributions and Outline We make the ollowin contributions: 1. The naive unctorial data model [8], as sketched above, is not quite appropriate or many practical inormation manaement tasks. Intuitively, this is because many cateorical constructions are only deined up to isomorphism; at a practical level, this means that every instance must contain only meaninless IDs [16]. The idea to extend the unctorial data model with attributes to capture meaninul concrete data such as strins and inteers was oriinally developed in [16]. In this paper, we reine that notion o attribute so that our schemas become special kinds o entityrelationship (ER) diarams (those in cateorical normal orm ), and our instances can be represented as relational tables that conorm to such ER diarams. (Sections 2 and 3) 2. We deine an alebraic query lanuae QL in which every query denotes a data miration unctor in the above extended sense. We show that QL queries are closed under composition, and how every query in QL can be described as a triplet o raph correspondences (similar to schema mappins [10]). Determinin whether a triplet o arbitrary raph correspondences is an QL query is semi-decidable. (Section 4) 3. We provide a translation o QL into the SPCU relational alebra o selection, projection, cartesian product, and union extended with a lobally unique ID enerator that constructs N + 1-ary tables rom N-ary tables by addin a lobally unique ID to each row. This allows us to enerate SQL prorams that implement QL. A corollary is that materializin result instances o QL queries has polynomial time data complexity. (Section 5) 4. We show that every SPCU query under ba semantics is expressible in QL, and how to extend QL with an operation allowin it to capture every SPCU query under set semantics. (Section 6) or every schema S, the cateory S Inst is a topos [4]. Hence, the unctorial data model admits other operations on instances besides the unctorial data miration operations that are the ocus o this paper. We conclude with a discussion o which o these operations can be implemented in SPCU+keyen (Section 7). 2 Cateorical Data In this section we deine the oriinal sinatures and instances o [16], as well as typed sinatures and typed instances, which are our extension o [16] to attributes. The basic idea is that sinatures are stylized ER diarams that denote cateories, and our database instances can be represented as instances o such ER diarams, and vice versa (up to natural isomorphism). rom this point on, we will distinuish between a sinature, which is a inite presentation o a cateory, and a schema, which is the cateory a sinature denotes. 2.1 Sinatures The unctorial data model [16] uses directed labeled multi-raphs and path equalities or sinatures. A path p is deined inductively as: p ::= node p.ede A sinature S is a inite presentation o a cateory. That is, a sinature S is a triple S := (N, E, C) where N is a inite set o nodes, E is a inite set o directed edes, and C is a inite set o path equations. or example: manaer Emp worksin secretary Dept Emp.manaer.worksIn = Emp.worksIn Dept.secretary.worksIn = Dept Here we see a sinature S with two nodes, three edes, and two path equations. This inormation enerates a cateory S: the ree cateory on the raph, modulo the equivalence relation induced by the path equations. The cateory S is the schema or S, and database instances over S are unctors S Set. Every path p: X Y in a sinature S denotes a morphism p: X Y in S. Given two paths p 1, p 2 in a sinature S, we say that they are equivalent, written p 1 p 2, i p 1 and p 2 are the same morphism in S. Two sinatures S and T are isomorphic, written S T, i they denote isomorphic schemas, i.e. i the cateories they enerate are isomorphic. 2

3 2.2 Cyclic Sinatures I a sinature contains a loop, it may or may not denote a cateory with ininitely many morphisms. Hence, some constructions over sinatures may not be computable. Testin i two paths in a sinature are equivalent is known as the word problem or cateories. The word problem can be semi-decided usin the completion without ailure extension [3] o the Knuth-Bendix alorithm. This alorithm irst attempts to construct a stronly normalizin re-write system based on the path equalities; i it succeeds, it yields a linear time decision procedure or the word problem [11]. I a sinature denotes a inite cateory, the Carmody-Walters alorithm [6] will compute its denotation. This alorithm computes let Kan extensions and can be used or many purposes in computational cateory theory [8]. 2.3 Instances Let S be a sinature. A S-instance is a unctor rom S to the cateory o sets. We will represent instances as relational tables usin the ollowin binary ormat: To each node N corresponds an identity or entity table named N, a relexive table with tuples o the orm (x, x). We can speciy this usin irstorder loic: xy.n(x, y) x = y. (1) The entries in these tables are called IDs or keys, and or the purposes o this paper we require them to be lobally unique. We call this the lobally unique key assumption. Note that it is possible to use unary tables instead o binary tables, but we have ound the uniorm binary ormat to be simpler when manipulatin instances usin e.., SQL. To each ede e: N 1 N 2 corresponds a link table e between identity tables N 1 and N 2. The axioms below say that every ede e: N 1 N 2 desinates a total unction N 1 N 2 : xy. e(x, y) N 1 (x, x) xy. e(x, y) N 2 (y, y) xyz. e(x, y) e(x, z) y = z x. N 1 (x, x) y.e(x, y) An example instance on our employees schema is: manaer Emp Emp Emp Emp Emp worksin Emp Dept 101 q q x02 Dept Dept Dept q10 q10 x02 x02 secretary Dept Emp x q (2) To save space, we will sometimes present instances in a joined ormat: Emp Emp manaer worksin q q x02 Dept Dept secretary q x The natural notion o equivalence o instances is isomorphism. In particular, the actual constants in the above tables should be considered meaninless IDs [2]. 2.4 Homomorphisms Let S be a sinature and I, J : S Set be S-instances. A database homomorphism h : I J is a natural transormation rom I to J: or every node N in S, h N is a unction rom the IDs in I(N) to the IDs in J(N) such that or every path p : X Y in S, we have h Y I(p) = J(p) h X. Homomorphisms can be represented as binary tables; or example, the identity homomorphism in our employees example is: Emp Emp Emp Attributes Dept Dept Dept q10 q10 x02 x02 Sinatures and instances, as deined above, do not have enouh structure to be useul in practice. At a practical level, we usually need ixed atomic domains like Strin to store actual data. We cannot simply include a node named, e.., Strin in a sinature because instances must be compared up to isomorphism o IDs but not isomorphism o strins. Hence, in this section we extend the unctorial data model by addin attributes. Let S be a sinature. A typin or S is a triple (A, i, γ) where A is a discrete cateory (cateory containin only objects and identity morphisms) consistin o attribute names, i is a unctor rom A to S mappin each attribute to its correspondin node, and γ is a unctor rom A to Set, mappin each attribute to its type (e.., the set o strins, the set o inteers). A γ i Set S Borrowin rom ER-diaram notation, we will write attributes as open circles and omit types. or example, we 3

4 can enrich our previous sinature with a typin (where each attribute has type Strin) as ollows: manaer Name Emp LName worksin secretary Emp.manaer.worksIn = Emp.worksIn Dept.secretary.worksIn = Dept Dept DName Importantly, and unlike [16], paths may not reer to attributes; they may only reer to nodes and edes. 2.6 Typed Instances Let S be a sinature and (A, i, γ) a typin. A typed instance I is a pair (I, δ) where I : S Set is an (untyped) instance toether with a natural transormation δ : I i γ. Intuitively, δ associates an appropriately typed constant (e.., a strin) to each ID in I : A γ i δ Set I S We represent the δ-part o a typed instance I as a set o binary tables as ollows: To each node N in S and attribute a i 1 (N) corresponds a binary attribute table mappin each ID in I(N) to a value in γ(a). In our employees example, we miht add the ollowin: Name Emp Strin 101 Alan 102 Andrey 103 Camille LName Emp Strin 101 Turin 102 Markov 103 Jordan DName Dept Strin x02 Math q10 CS Typed instances orm a cateory, and two instances are equivalent, written, when they are isomorphic objects in this cateory. Isomorphism o typed instances captures our expected notion o equality on typed instances, where the structure parts are compared or isomorphism and the attribute parts are compared or equality under such an isomorphism. This notion o equivalence o instances is the same notion as bein equivalent up to object identity in semantic data models (chapter 11 o [1]). 2.7 Typed Homomorphisms Database homomorphisms extend to typed instances. Let S be a typed sinature and I, J be S-instances. A database homomorphism h: I J is still, or each node N in S, a unction rom I(N) to J(N), obeyin naturality conditions. However, we also require that h must preserve attributes, so, or example, i (i, Alice) I(Name) then (h(i), Alice) J(Name). This deinition o database homomorphism coincides with the traditional deinition o database homomorphism rom database theory [1], where our IDs play the role o labelled nulls. 3 unctorial Data Miration In this section we deine the oriinal sinature morphisms and data miration unctors o [16], as well as typed sinature morphisms and typed data miration unctors, which are our extension o [16] to attributes. The basic idea is that associated with a morphism between sinatures : S T is a data transormation : T Inst S Inst and let and riht adjoints Σ, Π : S Inst T Inst. 3.1 Sinature Morphism Let S and T be sinatures. A sinature morphism : S T is a mappin that takes nodes in S to nodes in T and edes in S to paths in T ; in so doin, it must respect ede sources, ede tarets, and path equivalences. In other words, i p 1 p 2 is a path equivalence in S, then (p 1 ) (p 2 ) must be a path equivalence in T. Each sinature morphism : S T determines a unique schema morphism (indeed, a unctor) : S T in the obvious way. Two sinature morphisms 1 : S T and 2 : S T are equivalent, written 1 2, i they denote naturally isomorphic unctors. Below is an example sinature morphism. N1 A1 A2 N2 B1 N In the above example, the nodes N1 and N2 are mapped to N, the two morphisms to A1 are mapped to the morphism to B1, and the two morphisms to A2 are mapped to the morphism to B2. B2 4

5 3.2 Typed Sinature Morphisms Intuitively, sinature morphisms are extended to typed sinatures by providin an additional (node and typerespectin) mappin between attributes. or example, we miht have Name and Ae attributes in our typins and map Name to Name and Ae to Ae: N1 Name A1 A2 Ae N2 Name B1 N ormally, let S be a sinature and Γ := (A, i, γ) a typin or S. Let S be a sinature and Γ := (A, i, γ ) a typin or S. A typed sinature morphism rom (S, Γ) to (S, Γ ) consists o a sinature morphism : S S and a unctor (i.e. a morphism o typins ) G : A A such that the ollowin diaram commutes: G A A γ γ i Set i S S 3.3 Data Miration unctors Each sinature morphism : S T is associated with three data miration unctors,, Σ, and Π, which mirate instance data on S to instance data on T and vice versa. A summary is in iure 1 on the next pae. Deinition 1 (Data Miration unctors). Let : S T be a (untyped) sinature morphism. Deine : T Inst S Inst as: (I) := I : S Set S B2 Ae T (I) I Set and deine Σ, Π : S Inst T Inst to be let and riht adjoint to, respectively. Data miration unctors extend to typed data miration unctors over typed instances in a natural way. An example typed data miration is shown in iure 2 on the next pae. A similar example, but with a sinature that contains an ede, is shown in iure Data Miration on Homomorphisms So ar, we have only shown examples o the object, or instance to instance, part o the, Σ, and Π data miration operations. Because these data miration operations are unctors, they also posses a morphism, or database homomorphism to database homomorphism part. Rather than ive examples o these operations, we simply summarize that the unctorial data model admits the ollowin operations, or any : C D. I I, J D Inst and h : I J is a database homomorphism, then (h) : (I) (J). I I, J C Inst and h : I J is a database homomorphism, then Σ (h) : Σ (I) Σ (J) and Π (h) : Π (I) Π (J). 4 QL The oal o this section is to deine and study an alebraic query lanuae where every query denotes a composition o data miration unctors. Our syntax or queries is desined to build-in certain syntactic restrictions discussion in this section, and to provide a convenient normal orm. Deinition 2 (QL Query). A (typed) QL query Q rom S to T, denoted Q: S T, is a triple o typed sinature morphisms (, G, H): such that S S G S H T 1. S, S, S, and T are inite 2. G s attribute mappin is a bijection H is a discrete op-ibration [4] Semantically, the query Q: S T corresponds to a unctor Q: S Inst T Inst (indeed, a unctor takin inite instances to inite instances) iven as ollows: Q := Σ H Π G By choosin two o, G, and H to be the identity, we can recover, Σ, and Π. However, roupin, Σ, and Π toether like this ormalizes a query as a disjoint union o a join o a projection. Because and Π commute, we could also ormalize a query as a disjoint union o a projection o a join. The three conditions above are crucial to ensurin both that QL can be implemented on SPCU+keyen, and or QL queries to be closed under composition: 2 All results in this paper should also apply to surjections, but we have only proved the bijective case. 5

6 iure 1: Data miration unctors induced by a sinature morphism : S T Name Symbol Type Idea o deinition Relational partner Pullback : T Inst S Inst Composition with Project Riht Pushorward Π Π : S Inst T Inst Riht adjoint to Join Let Pushorward Σ Σ : S Inst T Inst Let adjoint to Union iure 2: Data miration example N1 Name Salary Ae N2 N Name Salary Ae N1 ID Name Salary 1 Alice $100 2 Bob $250 3 Sue $300 Σ N2 ID Ae N ID Name Salary Ae a Alice $100 null 1 b Bob $250 null 2 c Sue $300 null 3 d null 4 null 5 20 e null 6 null 7 20 null 8 null 9 30 Π N ID Name Salary Ae a Alice $ b Bob $ c Sue $ N ID Name Salary Ae a Alice $ b Alice $ c Alice $ d Bob $ e Bob $ Bob $ Sue $ h Sue $ i Sue $ iure 3: Data miration example with orein keys Name Salary N1 Ae N2 N Name Salary Ae N1 ID Name Salary 1 Alice $ Bob $ Sue $300 6 N2 ID Ae Σ Π N ID Name Salary Ae a Alice $ b Bob $ c Sue $

7 1. Condition 1 ensures that path equivalence in each sinature is in act decidable. Moreover, the iber product o two (ininite) initely-presented cateories may not be initely presented, so it may not be possible to compute composition queries when schemas are ininite. initeness o schemas also ensures that Π operations always ive inite answers. When the taret schema is ininite, Π may create uncountably ininite result instances. Consider the unique sinature morphism: S := s s =: T Here T has morphisms { n n N} so it is ininite. Given the two-element instance I : S Set with I(s) = {Alice, Bob}, the rowset in Π (I)(s) is the (uncountable) set o ininite streams in {Alice, Bob}, i.e. Π (I)(s) I(s) N. 2. Condition 2 ensures that the QL query is domain independent [1]; i.e., that constants resultin rom any Π operation are contained in the input instance. Consider the unique morphism: S := s s a (Int) =: T Given the one-element instance I : S Set with I(s) = {Alice}, the rowset in Π (I)(a) will contain, or every inteer i, a row (Alice i, i), i.e., Π (I)(a) {(Alice 1, 1), (Alice 2, 2),...}. 3. Condition 3 ensures union-compatibility [1], which allows us to implement Σ usin SPCU s union operation. A unctor : C D is called a discrete op-ibration i, or every object c C and every morphism : d d in D with (c) = d, there exists a c and unique morphism ḡ : c c in C such that (ḡ) =. or example, the ollowin unctor, which maps as to A, bs to B, etc. is a discrete op-ibration: C := D := b 1 b 2 B G a 1 a 2 a 3 h 1 h 2 h 3 c 1 c 2 c 3 c 4 A H C Intuitively, : C D is a discrete op-ibration i the columns in each table T o D are exactly matched by the columns in each table in C mappin to T. Since a 1, a 2, a 3 A, they must have the same column structure and they do: each has two non-id columns. Similarly each o the b i and each o the c i have no non-id columns, just like their imaes in D. Remark. Σ is deined or any, not just or discrete op-ibrations. However, such unrestricted Σs cannot be implemented usin SPCU+keyen (even i we allow OUTER UNION), and QL queries that use such Σs are not closed under composition. 4.1 Composition Theorem 1 (Closure under composition). Given any (typed) QL queries : S X and : X T, we can compute an QL query : S T such that Sketch o proo. ollowin the proo in [9, Proposition 1.1.2], it suices to show the Beck-Chevalley conditions hold or and its adjoints, and that dependent sums distribute over dependent products when the ormer are indexed by discrete op-ibrations. The proo and alorithm are iven in the appendix as Theorem Detailin the alorithm involves a number o constructions we have not deined, so we only ive a brie sketch here. To compose Σ t Π s with Σ v Π u we must construct the ollowin diaram: s B m n N (iv) B r A (iii) A S T U irst we orm the pullback (i). Then we orm the typed distributivity diaram (ii) as in Proposition Then we orm the typed comma cateories (iii) and (iv) as in Proposition The result then ollows rom Proposition 9.2.4, 9.2.5, and Corollary The composed query is Σ v w Π q p s m n. We have implemented this composition alorithm in our prototype QL IDE. h t p (i) e k u D D q (ii) M C w v 7

8 5 QL in SPCU+keyen In this section we let : S T be a typed sinature morphism and we deine SPCU+keyen translations or, Σ, and Π. By an SPCU+keyen proram we mean a list o SQL expressions, where each SQL expression is one o the ollowin: 1. A conjuctive query over base tables; i.e., a SQL expression o the orm SELECT DISTINCT - ROM - WHERE A union o SPCU-keyen expressions; i.e., a SQL expression - UNION A lobally-unique ID eneration operation (keyen) that appends to a iven table a new column with lobally unique IDs. To implement keyen in SQL, we irst initialize a lobal variable, usin, or example, the ollowin MySQL code: := 0. Then, to implement keyen we increment this lobal variable. or example, to add a column c o new unique IDs to a table t with one column called col, we would enerate: SELECT AS c ROM t 4. A CREATE TABLE ollowed by an INSERT INTO o an SPCU-keyen expression. A SQL/SPCU+keyen proram that implements a unctorial data miration S Inst T Inst operates over a SQL schema containin one binary table or each node, ede, and attribute in S, and stores one binary table or each node, ede, and attribute in T. 5.1 Translatin Theorem 2. We can compute a SPCU+keyen proram [ ] such that or every T -instance I T Inst, we have (I) [ ] (I). Proo. See Construction We sketch the alorithm as ollows. We are iven a T -instance I, presented as a set o unctional binary tables, and are tasked with creatin the S-instance (I). We describe the result o (I) on each table in the result instance by examinin the schema S: or each node N in S, the binary table (N) is the binary table I( (N)). or each attribute A in S, the binary table (A) is the binary table I( (A)). Each ede e : X Y in S maps to a path (e) : (X) (Y ) in T. We compose the binary edes tables makin up the path (e), and that becomes the binary table (e). The SQL eneration alorithm or above (and Σ, below) does not maintain the lobally unique ID requirement. or example, can copy tables. Hence we must also enerate SQL to restore the unique ID invariant. 5.2 Translatin Σ Theorem 3. Suppose is a discrete op-ibration. Then we can compute a SPCU+keyen proram [ ] Σ such that or every S-instance I S Inst, Σ (I) [ ] Σ (I). Proo. See Construction We sketch the alorithm as ollows. We are iven a S-instance I, presented as a set o unctional binary tables, and are tasked with creatin the T -instance Σ (I). We describe the result o Σ (I) on each table in the result instance by examinin the schema T : or each node N in T, the binary table Σ (N) is the union o the binary node tables in I that map to N via. or each attribute A in T, the binary table Σ (A) is the union o the binary attribute tables in I that map to A via. Let e : X Y be an ede in T. We know that or each c 1 (X) there is at least one path p c in S such that (p c ) e. Compose p c to a sinle binary table, and deine Σ (e) to be the union over all such c. The choice o p c will not matter. 5.3 Translatin Π Theorem 4. Suppose S and T are inite, and has a bijective attribute mappin. Then we can compute a SPCU+keyen proram [ ] Π such that or every S-instance I S Inst, we have Π (I) [ ] Π (I). Proo. See Construction The alorithm or Π is more complex than or and Σ. It makes use o comma cateories, which we have not deined, as well as limit tables, which are a sort o join all. We deine these now. Let B be a typed sinature and H a typed B- instance. The limit table lim B is computed as ollows. irst, take the cartesian product o every node table in B, and naturally join the attribute tables o B. Then, or each ede e : n 1 n 2 ilter the table by n 1 = n 2. This iltered table is the limit table lim B. Let S : A C and T : B C be unctors. The comma cateory (S T ) has or objects triples (α, β, ), with α an object in A, β an object in B, and : S(α) T (β) a morphism in C. The morphisms rom (α, β, ) to (α, β, ) are all the pairs (, h) where : α α and h : β β are morphisms in A and B respectively such that T (h) = S(). 8

9 The alorithm or Π proceeds as ollows. irst, or every object d T we consider the comma cateory B d := (d ) and its projection unctor q d : (d ) C. (Here we treat d as a unctor rom the empty cateory). Let H d := I q d : B d Set, constructed by eneratin SQL or qd (I). We say that the limit table or d is lim Bd H d, as described above. Now we can describe the taret tables in T : or each node N in T, enerate lobally unique IDs or each row in the limit table or N. These IDs are Π (N). or each attribute A o N in T, Π (A) will be a projection rom the limit table or N. or each ede e: X Y in T, Π (e) will be a map rom X to Y iven by joinin the limit tables or X and Y on columns which actor throuh e. Remark. Our SPCU+keyen eneration alorithms or and Σ work even when S and T are ininite, but this is not the case or Π. But i we are willin to take on ininite SQL queries, we have Π in eneral; see Construction An interestin direction or uture work is to relate these ininite SQL queries to recursive lanuaes such as Datalo. 5.4 Translatin unctorial Data Mirations o Homomorphisms We can also enerate SPCU+keyen to implement data mirations o homomorphisms (c.. section 3.4). Theorem 5. or all T -instances I, J T Inst and homomorphisms h : I J, we can compute a SPCU+keyen proram [ ] such that (h) [ ] (h), and similarly or Σ, Π. Proo. See Constructions 8.2.1, , and SPCU in QL To implement SPCU usin QL, we must encode relational schemas as sinatures with a sinle active domain [1] attribute. or example, a schema with two relations R(c 1,..., c n ) and R (c 1,..., c n ) becomes: R c 1 c n D A c n R c 1 We miht expect that the c 1,..., c n would be attributes o node R, and hence there would be no node D, but that does not work: attributes may not be (directly) joined on. Instead, we must think o each column o R as a mappin rom R s domain to IDs in D, and A as a mappin rom IDs in D to constants. Note that A need not be injective; our constructions below will not in eneral maintain injectiveness o A (althouh injectiveness can be recovered usin the dedup operation deined in the next section). We will write [R] or the encodin o a relational schema R and [I] or the encodin o a relational R-instance I. 6.1 SPCU in QL (Ba Semantics) QL can implement SPCU queries under ba semantics directly usin the above encodin. In what ollows we will omit the attribute A rom the diarams. We may express the (ba) operations π, σ,, + as ollows: Let R be a table. We can express π i1,...,i k R as, where is any unctor sendin πr to R and D to D in the ollowin diaram: i 1 πr i k D c 1 R c n D This construction is only appropriate or ba semantics because πr will have the same number o rows as R. Let R be a table. We can express σ a=b R as Π, where is: a c 1 R c n D b x c 1 σr c n D Here (a) = (b) = x and (c i ) = c i or 1 i n. Remark. In act, creates the duplicated column ound in σ a=b R. I we wanted the more economical query in which the column is not duplicated, we would use Π instead o Π. Let R and R be tables. We can express R R as Π, where is: R c 1 c n D c n R c 1 c 1 R R c n c 1 D c n 9

10 Let R and R be union-compatible tables. We can express R + R as Σ, where is : R c 1 c n D c n R c 1 c 1 R+R c n D Theorem 6 (SPCU in QL (Ba Semantics)). Let R be a relational schema, and I an R-instance. or every SPCU query q under ba semantics we can compute a QL query [q] such that [q(i)] [q]([i]). Proo. See Propositions and SPCU in QL (Set Semantics) The encodin stratey described above ails or SPCU under its typical set-theoretic semantics. or example, consider a simple two column relational table R, its encoded QL instance [R], and an attempt to project o col1 usin : R col1 col2 x y x z [R] ID col1 col2 0 x y 1 x z [R] ID col1 0 x 1 x This answer is incorrect under projection s settheoretic semantics, because it has the wron number o rows. However, it is possible to extend QL with an idempotent operation, dedup T : T Inst T Inst, such that QL+dedup can implement every SPCU query under set-theoretic semantics. Intuitively, dedup T converts a T -instance to a smaller T -instance by equatin IDs that cannot be distinuished by any attribute alon any path. In relational lanuaes based on object- IDs, this operation is usually called copy elimination [2]. The dedup o the above instance is: dedup( [R]) ID col1 0 x which is correct or the set-theoretic semantics. Theorem 7 (SPCU in QL (Set Semantics)). Let R be a relational schema, and I an R-instance. or every SPCU query q under set semantics we can compute a QL proram [q] such that [q(i)] dedup T ([q]([i])). Proo. See Propositions and Provided T is obtained rom a relational schema (i.e., has the pointed orm described in this section), dedup T can be implemented usin SPCU+keyen, and data mirations o the orm dedup Q, where Q is an QL query, are closed under composition. The QL IDE implements a (partial) translation SQL QL usin the above constructions. 7 Conclusion The unctorial data model admits other operations on instances and homomorphisms besides the, Σ, Π data miration unctors described in this paper. In cateorical terms, or every untyped schema S, the cateory o S instances is a topos (model o hiher-order loic) [4], and or every inite typed schema S with inite attributes (e.., Strin is not allowed, but VARCHAR(256) is), the cateory o inite S instances is a topos. The QL IDE implements n-ary products and co-products o instances by translatin to SPCU+keyen. Exponential and subobject classiier instances cannot be implemented usin SQL, but the QL IDE can execute them directly. Reerences [1] Sere Abiteboul, Richard Hull, and Victor Vianu. oundations o Databases. Addison-Wesley, [2] Sere Abiteboul and Paris C. Kanellakis. Object identity as a query lanuae primitive. In Proceedins o the 1989 ACM SIGMOD International Conerence on Manaement o Data, SIGMOD 89, paes , New York, NY, USA, ACM. [3] Leo Bachmair, Nachum Dershowitz, and David A. Plaisted. Completion without ailure, [4] Michael Barr and Charles Wells, editors. Cateory theory or computin science, 2nd ed [5] P. Buneman, S. Davidson, and A. Kosky. Theoretical aspects o schema merin. In EDBT, [6] S. Carmody, M. Leemin, and R..C. Walters. The toddcoxeter procedure and let kan extensions. Journal o Symbolic Computation, 19(5): , [7] Ronald ain, Phokion G. Kolaitis, Lucian Popa, and Wan- Chiew Tan. Composin schema mappins: Second-order dependencies to the rescue. ACM Trans. Database Syst., 30(4): , December [8] Michael lemin, Ryan Gunther, and Robert Rosebruh. A database o cateories. Journal o Symbolic Computin, 35: , [9] N. Gambino and J. Kock. Polynomial unctors and polynomial monads. [10] Laura M. Haas, Mauricio A. Hernández, Howard Ho, Lucian Popa, and Mary Roth. Clio rows up: rom research prototype to industrial tool. In SIGMOD 05. [11] Jieh Hsian and Michael Rusinowitch. On word problems in equational theories. In Automata, Lanuaes and Prorammin, volume 267 o LNCS [12] Bart Jacobs. Cateorical loic and type theory. PhD thesis, Mathematics, Amsterdam, Lausanne, New York, [13] Michael Johnson and Robert Rosebruh. Sketch data models, relational schema and data speciications. Electronic Notes in Theoretical Computer Science, 61(0):51 63, [14] Peter T. Johnstone. Sketches o an elephant: a topos theory compendium. Vol [15] David I Spivak. Database queries and constraints via litin problems [16] David I. Spivak. unctorial data miration. In. Comput., 217:31 51, Auust [17] Limsoon Won. Queryin nested collections. PhD thesis, Philadelphia, PA, USA, AAI

11 8 Cateorical constructions or data miration Here we ive many standard deinitions rom cateory theory and a ew less-thanstandard (or oriinal) results. The proos oten assume more knowlede than the deinitions do. Let us also note our use o the term essential, which basically means up to isomorphism. Thus an object X havin a certain property is essentially unique i every other object havin that property is isomorphic to X; an object X is in the essential imae o some unctor i it is isomorphic to an object in the imae o that unctor; etc. 8.1 Basic constructions Deinition (iber products o sets). Suppose iven the diaram o sets and unctions as to the let in (3). B A C B A C A h B C (3) Its iber product is the commutative diaram to the riht in (3), where we deine A C B := {(a, c, b) (a) = c = (b)} and,, and h = = are the obvious projections (e.. (a, c, b) = b). We sometimes reer to just (A C B,,, h) or even to A C B as the iber product. Example (Chain sinatures). Let n N be a natural number. The chain sinature on n arrows, denoted n, 3 is the raph 0 1 n with no path equivalences. One can check that n has ( ) n+2 2 -many paths. The chain sinature 0 is the terminal object in the cateory o sinatures. A sinature morphism c: 0 C can be identiied with an object in C. A sinature morphism : 1 C can be identiied with a morphism in C. It is determined up to path equivalence by a pair (n, p) where n N is a natural number and p: n C is a morphism o raphs. Deinition (iber product o sinatures). Suppose iven the diaram o sinatures and sinatures mappins as to the let in (4). B A C B A C A h B C (4) 3 The chain sinature n is oten denoted [n] in the literature, but we did not want to urther overload the bracket [-] notation 11

12 On objects and arrows we have the ollowin diarams o sets: Ob(B) Ob() Ob(A) Ob() Ob(C) Arr(B) Arr() Arr(A) Arr() Mor(C) (5) We deine the iber product o the let-hand diaram in (4) to be the riht-hand commutative diaram in (4), where we the objects and arrows o A C B are iven by the iber product o the let and riht diaram o sets in (5), respectively. A pair o paths in A C B are equivalent i they map to equivalent paths in both A and B (and thereore in C). Note that we have iven A C B as a sinature, i.e. we have provided a inite presentation o it. Example (Pre-imae o an object or morphism). Let : C D be a unctor. Given an object d Ob(D) (respectively a morphism d Mor(D)), we can reard it as a unctor d: 0 D (respectively a unctor d: 1 D), as in Example The pre-imae o d is the subcateory o C that maps entirely into d. This is the iber product o the diaram C D d 0 (respectively the diaram C D d 1 ). A sinature is assumed to have initely many arrows, but because it may have loops, C may have ininitely many non-equivalent paths. We make the ollowin deinitions to deal with this. Deinition A sinature C is called inite i there is a inite set S o paths such that every path in C is equivalent to some s S. An equivalent condition is that the schema C has initely many morphisms. (Note that i C is inite then C has initely many objects too, since each object has an associated trivial path.) Recall (rom the top o Section 2) that every sinature is assumed to have a inite set o objects and arrows. The above notion is a semantic one: a sinature C is called inite i C is a inite cateory. Deinition A sinature C is called acyclic i it has no non-trivial loops. In other words, or all objects c Ob(C) every path p: c c is a trivial path. Lemma (Acyclicity implies initeness). I C is an acyclic sinature then it is inite. Proo. All o our sinatures are assumed to include only initely many objects and arrows. I C is acyclic then no ede can appear twice in the same path. Suppose that C has n arrows. Then the number o paths in C is bounded by the number o lists in n letters o lenth at most n. This is inite. Lemma (Acyclicity and initeness are preserved under ormation o iber product). Given a diaram A C G B o sinatures such that each is acyclic (respectively, inite), the iber product sinature A C B is acyclic (respectively, inite). Proo. The iber product o inite sets is clearly inite, and the iber product o cateories is computed by twice takin the iber products o sets (see Deinition 8.1.3). We now show that acyclicity is preserved. 12

13 Let Loop denote the sinature Loop := (6) s A cateory C is acyclic i and only i every unctor L: Loop C actors throuh the unique unctor Loop 0, i.e. i and only i every loop is a trivial path. The result ollows by the universal property or iber products. Deinition (Comma cateory). To a diaram o cateories A C B we can associate the comma cateory o over, denoted ( ), deined as ollows. Ob( ) := {(a, b, γ) a Ob(A), b Ob(B), γ : (a) (b) in C} Hom ( ) ((a, b, γ), (a, b, γ )) := {(m, n) m: a a in A, n: b b in B, γ (m) = (n) γ} There are obvious projection unctors ( ) p A and ( ) q B and a natural transormation α: p q, and we summarize this in a canonical natural transormation diaram ( ) q B (7) p A α C Below is a heuristic picture that may be helpul. The outside square consists o cateories and unctors (note that it does not commute!). The inside square represents a morphism in ( ) rom object (a, b, γ) to object (a, b, γ ); this diaram is required to commute in C. p ( ) q A m a (a) (m) a (a ) γ (b) b (n) γ (b ) b n B C Example (Slice cateory and coslice cateory). Let : A B be a unctor, and let b Ob(B) be an object, which we represent as a unctor b: 0 B. The slice cateory o over b is deined as ( b). The coslice cateory o b over is deined as (b ). or example let { } denote a one element set, and let id Set be the identity unctor on Set. Then the coslice cateory ({ } id Set ) is the cateory o pointed sets, which we denote by Set. The slice cateory (id Set { }) is isomorphic to Set. 13

14 Lemma (Comma cateories as iber products). Given a diaram A C B, the comma cateory ( ) can be obtained as the iber product in the diaram ( ) α C 1 (p,q) A B (,) C C (dom,cod) where C 1 is the cateory o unctors 1 C. Proo. Straihtorward. Lemma (Acyclicity and initeness are preserved under ormation o comma cateory). Let A, B, and C be acyclic (resp. inite) cateories, and suppose we have unctors A C B. Then the comma cateory ( ) is acyclic (resp. inite). Proo. Since the iber product o inite sets is inite, Lemma implies that the ormation o comma cateories preserves initeness. Suppose that A and B are acyclic (in act this is enouh to imply that ( ) is acyclic). Given a unctor L: Loop ( ), this implies that composin with the projections to A and B yields trivial loops. In other words L consists o a diaram in C o the orm (a) γ (b) (a) γ (b) But this implies that γ = γ, so L is a trivial loop too, competin the proo. 8.2 Data miration unctors Construction ( ). Let : C D be a unctor. We will deine a unctor : D Inst C Inst; that is, iven a D-instance I : D Set we will naturally construct a C-instance (I). Mathematically, this is obtained simply by composin the unctors to et (I) := I : C Set C D I Set I To understand this on the level o binary tables, we take and I as iven and proceed to deine the C-instance J := I as ollows. Given an object c Ob(C), it is sent to an object d := (c) Ob(D), and I(d) is an entity table. Assin J(c) = I(d). Given an arrow : c c in C, it is sent to a path p = () in D. We can compose this to a binary table [p]. Assin J() = [p]. It is easy to check that these assinments preserve the axioms. 14

15 Lemma (initeness is preserved under ). Suppose that C and D are inite cateories and : C D is a unctor. I I : D Set is a inite D-instance then I is a inite C-instance. Proo. This ollows by construction: every object and arrow in C is assined some inite join o the tables associated to objects and arrows in D, and inite joins o inite tables are inite. Recall the deinition o discrete op-ibrations rom Deinition??. Lemma (Discrete op-ibrations). Let : C D be a sinature morphism. The ollowin are conditions on are equivalent: 1. The unctor [ ]: [C] [D] is a discrete op-ibration. 2. or every choice o object c 0 in C and path q : d 0 d n in D with (c 0 ) = d 0, there exists a path p C such that (p) q, and p is unique up to path equivalence. 3. or every choice o object c 0 in C and ede q : d 0 d 1 in D with (c 0 ) = d 0, there exists a path p C such that (p) q, and p is unique up to path equivalence. Proo. It is clear that condition (2) implies condition (3), and the converse is true because concatenations o equivalent paths are equivalent. It suices to show that (1) and (2) are equivalent. It is shown in [15] that a unctor π : X S between cateories X and S is a discrete op-ibration i and only i or each solid-arrow square o the orm c 0 0 X i 0 1!p q where i 0 : 0 1 sends the unique object o 0 to the initial object o 1, there exists a unique unctor l such that both trianles commute. We now translate this statement to the lanuae o sinatures with C = X and D = S. A unctor 1 C corresponds to a path in C, and the uniqueness o lit p corresponds to uniqueness o equivalence class. This completes the proo. Example (A discrete op-ibration). Let C, D be sinatures, and let : C D be as suested by the picture below. S, b 1 1 a 1 h 1 c 1 π C := 2 b 2 a 2 3 a 3 h 2 h 3 c 2 c 3 c 4 15

16 D := B G A H C This is a discrete op-ibration. Corollary (Pre-imae o an object under a discrete op-ibration is discrete). Let : C D be a discrete op-ibration and let d Ob(D) be an object. Then the pre-imae 1 (d) is a discrete sinature (i.e. every path in 1 (d) is equivalent to a trivial path). Proo. or any object d Ob(D) the pre-imae 1 (d) is either empty or not. I it is empty, then it is discrete. I 1 (d) is non-empty, let e: c 0 c 1 be an ede in it. Then (e) = d = (c 0 ) so we have an equivalence e c 0 by Lemma Example Let : A B be a unctor. or any object b B, considered as a unctor b: 0 B, the induced unctor (b ) B is a discrete op-ibration. Indeed, iven an object b (a) in (b ) and a morphism h: a a in A, there is a unique map b (a) (h) (a ) over it. Deinition A sinature C is said to have non-redundant edes i it satisies the ollowin condition or every ede e and path p in C: I e p, then p has lenth 1 and e = p. Proposition Every acyclic sinature is equivalent to a sinature with non-redundant edes. Proo. The important observation is that i C is acyclic then or any ede e and path p in C, i e is an ede in p and e p then e = p. Thus, we know that i C is acyclic and e p then e is not in p. So enumerate the edes o C as E C := {e 1,..., e n }. I e 1 is equivalent to a path in E C {e 1 } then there is an equivalence o sinatures C C {e 1 } =: C 1 ; i not, let C 1 := C. Proceed by induction to remove each e i that is equivalent to a path in C i 1, and at the end no ede will be redundant. Corollary (Discrete op-ibrations and preservation o path lenth). I C and D have non-redundant edes and : C D is a discrete op-ibration, then or every choice o object c 0 in C and path q = d 0.e 1.e e n o lenth n in D with (c 0 ) = d 0, there exists a unique path p = c 0.e 1.e e n o lenth n in C such that (e i ) = e i or each 1 i n, so in particular (p) = q. Proo. Let c 0, d 0, and q be as in the hypothesis. We proceed by induction on n, the lenth o q. In the base case n = 0 then Corollary implies that every ede in 1 (d 0 ) is equivalent to the trivial path c 0, so the result ollows by the non-redundancy o edes in C. Suppose the result holds or some n N. To prove the result or n + 1 it suices to consider the inal ede o q, i.e. we assume that q = e is simply an ede. By Lemma there exists a path p C such that (p) e, so by non-redundancy in D we know that (p) has lenth 1. This implies that or precisely one ede e i in p we have (e i ) = e, and or all other edes e j in p we have (e j ) is a trivial path. But by the base case this implies that p has lenth 1, completin the proo. 16

17 Construction (Σ or discrete opibrations). Suppose that : C D is a discrete op-ibration. We can succinctly deine Σ : C Inst D Inst to be the let adjoint to, however the ormula has a simple description which we ive now. Suppose we are iven and an instance I : C Set, considered as a collection o binary tables, one or each object and each ede in C. We are tasked with indin a D-instance, J := Σ I : D Set. We irst deine J on an arbitrary object d Ob(D). By Corollary 8.2.5, the pre-imae 1 (d) is discrete in C; that is, it is equivalent to a inite collection o object tables. We deine J(d) to be the disjoint union J(d) := I(c). c 1 (d) Similarly, let e: d d be an arbitrary arrow in D. By Lemma we know that or each c 1 (d) there is a unique equivalence class o paths p c in C such that (p c ) e. Choose one, and compose it to a sinle binary table all other choices will result in the same result. Then deine J(e) to be the disjoint union J(e) := I(p c ). c 1 (d) Lemma (initeness is preserved under Σ). Suppose that C and D are inite cateories and : C D is a unctor. I I : C Set is a inite C-instance then Σ I is a inite D instance. Proo. or each object or arrow d in D, the table Σ I(d) is a inite disjoint union o composition-joins o tables in C. The inite join o inite tables is inite, and the inite union o inite tables is inite. Remark (Σ exists more enerally and perorms quotients and skolemization). or any unctor : C D the unctor : D Inst C Inst has a let adjoint, which we can denote by Σ : C Inst D Inst because it arees with the Σ constructed in in the case that is a discrete op-ibration. Certain queries are possible i we can use Σ in this more eneral case namely, quotientin by equivalence relations and the introduction o labeled nulls (a.k.a. Skolem variables). We do not consider it much in this paper or a ew reasons. irst, quotients and skolem variables are not part o the relational alebra. Second, the set o queries that include such quotients and skolem variables are not obviously closed under composition (see Section 7). Construction (Limit as a kind o join all ). Let B be a sinature and let H be a B-instance. The unctor [H]: [B] Set has a limit lim B H Ob(Set), which can be computed as ollows. orettin the path equivalence relations, axioms (1) and (2) imply that H consists o a set {N 1,..., N m } o node tables, a set {e 1,..., e n } o ede tables, and unctions s, t: {1,..., n} {1,..., m} such that or each 1 i n the table 17

18 e i constitutes a unction e i : N s(i) N t(i). or each i deine X i as ollows: X 0 := π 2,4,...,2m (N 1 N m ) X 1 := π 1,2,...m σ s(1)=m+1 σ t(1)=m+2 (X 0 e 1 ). X i := π 1,2,...m σ s(i)=m+1 σ t(i)=m+2 (X i 1 e i ). X n := π 1,2,...m σ s(n)=m+1 σ t(n)=m+2 (X n 1 e n ) Then X := X n is a table with m columns, and its set X o rows (which can be constructed i one wishes by concatenatin the ields in each row) is the limit, X lim B H. Construction (Π). Let : C D be a sinature morphism. We can succinctly deine Π : C Inst D Inst to be the riht adjoint to, however the ormula has an alorithmic description which we ive now. Suppose we are iven and an instance I : C Set, considered as a collection o binary tables, one or each object and each ede in C. We are tasked with indin a D-instance, J := Π I : D Set. We irst deine J on an arbitrary object d: 0 D (see Example 8.1.2). Consider the comma cateory B d := (d ) and its projection q d : (d ) C. Note that in case C and D are inite, so is B d. Let H d := I q d : B d Set. We deine J(d) := lim Bd H d as in Construction Now let e: d d be an arbitrary arrow in D. or ease o notation, rewrite B := B d, B := B d, q := q d, q := q d, H := H d, and H := H d. We have the ollowin diaram o cateories (d ) (e ) (d ) H q I C Set A unique natural map J(d) = lim B H lim B H = J(d ) is determined by the universal property or limits, but we ive an idea o its construction. There is a map rom the set o nodes in B to the set o nodes in B and or each node N in B the correspondin node tables in H and H aree. Let X i and X i be deined respectively or (B, H) and (B, H ) as in Construction It ollows that X 0 is a projection (or column duplication) on X 0. The set o edes in B also map to the set o edes in B and or each ede e in B the correspondin ede tables in H and H aree. Thus the select statements done to obtain J(d ) = X contains the set o select statements perormed to obtain J(d ) = X. Thereore, the map rom J(d) to J(d ) is iven as the inclusion o a subset ollowed by a projection. I C or D is not inite, then the riht pushorward Π o a inite instance I C Set may have ininite, even uncountable results. q H 18