A Compositional Distributional Model of Meaning

Transcription

1 Compositional Distributional Model o Meaning Stephen Clark ob Coecke Mehrnoosh Sadrzadeh Oxord University Computing Laboratory Wolson uilding, Parks Road, OX1 3QD Oxord, UK stephen.clark@comlab.ox.ac.uk coecke@comlab.ox.ac.uk ms6@ecs.soton.ac.uk bstract We propose a mathematical ramework or a uniication o the distributional theory o meaning in terms o vector space models, and a compositional theory or grammatical types, namely Lambek s semantics. key observation is that the monoidal category o (inite dimensional) vector spaces, linear maps and the tensor product, as well as any, are examples o compact closed categories. Since, by deinition, a is a compact closed category with trivial morphisms, its compositional content is relected within the compositional structure o any non-degenerate compact closed category. The (slightly reined) category o vector spaces enables us to compute the meaning o a compound well-typed sentence rom the meaning o its constituents, by liting the type reduction mechanisms o semantics to the whole category. These sentence meanings live in a single space, independent o the grammatical structure o the sentence. Hence we can use the inner-product to compare meanings o arbitrary sentences. variation o this procedure which involves constraining the scalars o the vector spaces to the semiring o ooleans results in the well-known Montague semantics. Introduction This paper combines, and then exploits, three results presented at the 2007 Quantum Interaction symposium. The irst author and Pulman proposed the use o the Hilbert space tensor product to assign a distributional meaning to sentences (Clark & Pulman 2007). The second author showed that the Hilbert space ormalism or quantum mechanics, when recast in category-theoretic terms, admits a true logic which enables automation (Coecke 2007). The third author showed that the logic o semantics or grammatical types is an essential ragment o this logic o the Hilbert space ormalism (Sadrzadeh 2007). The symbolic (Dowty, Wall, & Peters 1981) and distributional (Schuetze 1998) theories o meaning are somewhat orthogonal with competing pros and cons: the ormer is compositional but only qualitative, the latter is noncompositional but quantitative. 1 Following (Smolensky & 1 See (Gazdar 1996) or a discussion o the two competing paradigms in Natural Languge Processing. Legendre 2005) in the context o Cognitive Science, where a similar problem exists between the connectionist and symbolic models o mind, the irst author and Pulman argued or the use o the tensor product o vector spaces. They suggested that to implement this idea in linguistics one can, or example, traverse the parse tree o a sentence and tensor the vectors o the meanings o words with the vectors o their roles: ( John subj) likes ( Mary obj) This vector in the tensor product space should then be regarded as the meaning o the sentence John likes Mary. In this paper we will also pair vectors in meaning spaces and grammatical types, but in a way which overcomes some o the shortcomings o (Clark & Pulman 2007). One shortcoming is that, since inner-products can only be computed between vectors which live in the same space, sentences can only be compared i they have the same grammatical structure. In this paper we provide a procedure to compute the meaning o any sentence as a vector within a single space. second problem is the lack o a method to compute the vectors representing the grammatical type; the procedure presented here does not require such vectors. bramsky and the second author proposed a categorical quantum axiomatics as a high-level ramework to reason about quantum phenomena. Primarily this categorical axiomatics is a logic which lives on top o linear and multi-linear algebra and hence also applies to the use o vector space machinery outside the domain o quantum physics. The passage to the category-theoretic level allows or much simpler mathematical structures than vector spaces to exhibit quantum-like eatures. For example, while there is nothing quantum about sets, when organised in a category with relations (not unctions) as morphisms and cartesian product (not disjoint union) as tensor, many typical quantum eatures emerge (bramsky & Coecke 2004; Coecke 2005b). syntax or these more general pseudo quantum categories more precisely, tensorial matrix calculi over some semiring can be ound in (bramsky & Duncan 2006). In this paper we will exploit both the categorical logic which lives on top o vector spaces as well as the act that it also lives on a much simper relational variant. The use o s or analysing the structure o natural s is a recent development by (Lambek 1999)

2 which builds on the original Lambek calculus (Lambek 1958) where types are used to analyze the syntax o natural s in a simple algebraic setting. Pregroups have been used to analyze the syntax o many s, or example English (Lambek 2004), French (argelli & Lambek 2001b), rabic (argelli & Lambek 2001a), Italian (Casadio & Lambek 2001), and Persian (Sadrzadeh 2006). They have also been provided with a Montague-style semantics (Preller 2007) which equips them with a translation into the lambda calculus and predicate logic. s discussed in (Sadrzadeh 2007) s are posetal instances o the categorical logic o vector spaces as in (bramsky & Coecke 2004) where juxtaposition o types corresponds to the tensor product o the monoidal category o vector spaces, linear maps and the tensor product. The diagrammatic toolkit o noncommutative categorical quantum logic introduces reduction diagrams or typing sentences and allows the comparison o the grammatical patterns o sentences in dierent s (Sadrzadeh 2006). Here we blend these three ideas together, and articulate the result in a rigourous ormal manner. We provide a mathematical structure where the meanings o words are vectors in vector spaces, their grammatical roles are types in a, and tensor product is used or the composition o meanings and types. We will pass rom vector spaces to the category o vector spaces equipped with the tensor product, and reine this structure with the non-commutative aspects o the structure. Type-checking is now an essential ragment o the overall categorical logic. The vector spaces enrich these types with quantities: the reduction scheme to veriy grammatical correctness o sentences will not only provide a statement on the well-typedness o a sentence, but will also assign a vector in a vector space to each sentence. Hence we obtain a theory with both analysis and vector space models as constituents, but which is inherently compositional and assigns a meaning to a sentence given the meanings o its words. The vectors s representing the meanings o sentences all live in the same meaning space S. Hence we can compare the meanings o any two sentences s, t S by computing their inner-product s t. Surprisingly, Montague semantics emerges as a simpliied variant o our setting, by restricting the vectors to range over {0, 1}, where sentences are simply true or alse. Theoretically, this is nothing but the passage rom the category o vector spaces to the category o relations as described in (bramsky & Coecke 2004; Coecke 2005b). In the same spirit, one can look at vectors ranging over N or Q and obtain degrees or probabilities o meaning. s a inal remark, in this paper we only set up our general mathematical ramework and leave a practical implementation or uture work. Linguistic background We briely present two domains o Computational Linguistics which provide the linguistic background or this paper, and reer the reader to the literature or more details. 1. Vector space models o meaning. The key idea behind vector space models o meaning (Schuetze 1998) can be summed up by Firth s ot-quoted dictum that you shall know a word by the company it keeps. The basic idea is that the meaning o a word can be determined by the words which appear in its contexts, where context can be a simple n-word window, or the argument slots o grammatical relations, such as the direct object o the verb eat. Intuitively, cat and dog have similar meanings (in some sense) because cats and dogs sleep, run, walk; cats and dogs can be bought, cleaned, stroked; cats and dogs can be small, big, urry. This intuition is relected in text because cat and dog appear as the subject o sleep, run, walk; as the direct object o bought, cleaned, stroked; and as the modiiee o small, big, urry. Meanings o words can be represented as vectors in a high-dimensional meaning space, in which the orthogonal basis vectors are represented by context words. To give a simple example, i the basis vectors correspond to eat, sleep, and run, and the word dog has eat in its context 6 times (in some text), sleep 5 times, and run 7 times, then the vector or dog in this space is (6,5,7). The advantage o representing meanings in this way is that the vector space gives us a notion o distance between words, so that the inner product (or some other measure) can be used to determine how close in meaning one word is to another. Computational models along these lines have been built using large vector spaces (tens o thousands o context words/basis vectors) and large bodies o text (up to a billion words in some experiments). Experiments in constructing thesauri using these methods have been relatively successul. For example, the top 10 most similar nouns to introduction, according to the system o (Curran 2004), are launch, implementation, advent, addition, adoption, arrival, absence, inclusion, creation. 2. Pregroup semantics or grammatical types. Pregroup semantics was proposed by Lambek as a substitute or the Lambek Calculus (Lambek 1958), a well-studied type categorial grammar (Moortgat 1997). The main reason we chose the theory o semantics is its mathematical structure, which is ideal or the mathematical developments in this paper. The key ingredient is the mathematical object o a, which provides a tidy and simple algebraic structure to mechanically check i sentences o a are grammatical. partially ordered monoid is a triple (P,, ) where (P, ) is a partially ordered set, (P, ) is a monoid with unit 1, and or all a, b, c P with a b we have c a c b and a c b c. (1) is a partially ordered monoid (P,, ) where each type p P has a let adjoint p l and a right adjoint p r, which means that p, p l, p r P satisy p l p 1 p p l and p p r 1 p r p. From this it also ollows that 1 l 1 r Roughly speaking, the passage rom Lambek Calculus to Pregroups is obtained by replacing the two residuals o the juxtaposition operator (monoid multiplication) with two unary operations. This enables us to work with a direct encoding o unction arguments as adjoints rather than an indirect encoding o them through negation (implication by bottom).

3 Similar to type categorial grammars, one starts by ixing some basic grammatical roles and a partial ordering between them. We then reely generate a (P,,, ( ) l, ( ) r ) o these types where 1 is now the unit o juxtaposition, that is, the empty type. Examples o the basic types are: π or pronoun s or declarative statement q or yes-no question i or ininitive o the verb o or direct object. In cases where the person o the pronoun and tense o the verb matters, we also have π j, s k, q k P or j th person pronoun and k th tense sentence and question. We require the ollowing partial orders: π j π s k s q k q The adjoints and juxtapositions o these types are used to orm the compound types. type is assigned to each word in a sentence and the monoid multiplication is used or juxtaposition. The juxtaposition o adjacent adjoint types causes reduction to 1. This process is repeated until no more reduction is possible and a type is returned as the main type o the juxtaposition. I this type is the desired type (e.g. s or statement and q or question), the juxtaposition is a grammatical sentence. It has been shown in (uszkowski 2001) that this procedure is decidable. Thus we obtain a decision procedure to determine i a given sentence o a is grammatical. For simplicity, we use an arrow or and drop the between juxtaposed types. For the example sentence He likes her, we have the ollowing type assignment: He likes her π 3 (π r so l ) o or which we obtain the ollowing reduction: π 3 (π r so l )o π(π r so l )o 1s1 s The reduction can be represented diagrammatically by conjoining the adjacent adjoint types. For example, or the above reduction we have the ollowing diagram: π (π r s o l ) o The same method is used to analyze other types o sentences; or example, to type a yes-no question we assign (io l ) to the ininitive o the transitive verb and (qi l π l ) to does, and obtain the ollowing reduction or Does he like her? : Does he like her? question (qi l π l ) π 3 (io l ) o q The reduction diagrams are helpul in demonstrating the order o reductions, especially in compound sentences. They can also be used to compare the grammatical patterns o dierent s (Sadrzadeh 2006). Category-theoretic background We now sketch the mathematical prerequisites which constitute the ormal skeleton or the developments in this paper. Note that, in contrast to (bramsky & Coecke 2004), here we consider the non-symmetric case o a compact closed category, non-degenerate s being examples o essentially non-commutative compact closed categories. See (Freyd & Yetter 1989) or more details. Other reerences can be ound in (bramsky & Coecke 2004; Coecke 2005a; 2006). 1. Monoidal categories. The ormal deinition o monoidal categories is somewhat involved. It does admit an intuitive operational interpretation and an elegant, purely diagrammatic calculus. (strict) monoidal category C requires the ollowing data and axioms: a amily C o objects; or each ordered pair o objects (, ) a corresponding set C(, ) o morphisms; it is convenient to abbreviate C(, ) by : ; or each ordered triple o objects (,, C), and each : and g : C, there is a sequential composite g : C; we moreover require that: (h g) h (g ) ; or each object there is an identity morphism 1 : ; or : we moreover require that: 1 and 1 ; or each ordered pair o objects (, ) a composite object ; we moreover require that: 3 ( ) C ( C) ; there is a unit object I which satisies: 4 I I ; or each ordered pair o morphisms ( : C, g : D) a parallel composite g : C D; we moreover require biunctoriality i.e. (g 1 g 2 ) ( 1 2 ) (g 1 1 ) (g 2 2 ). There is a very intuitive operational interpretation o monoidal categories. We think o the objects as types o systems. We think o a morphism : as a process which takes a system o type (say, in some state ψ) as input and provides a system o type (say, in state (ψ)) 3 In the standard deinition o monoidal categories this strict equality is not required but rather the existence o a natural isomorphism between ( ) C and ( C). We assume strictness in order to avoid talking about natural transormations, and the corresponding coherence conditions between these natural isomorphisms. This simpliication is justiied by the act that each monoidal category is categorically equivalent to a strict one, which is obtained by imposing appropriate congruences. 4 gain we assume strictness in contrast to the usual deinition.

4 as output. Composition o morphisms is sequential application o processes. The compound type represents joint systems. We think o I as the trivial system, which can be either nothing or unspeciied. More on this intuitive interpretation can be ound in (Coecke 2006). In the graphical calculus or monoidal categories we depict morphisms by boxes, with incoming and outgoing wires labelled by the corresponding types, with sequential composition depicted by connecting matching outputs and inputs, and with parallel composition depicted by locating boxes side by side. For example, the morphisms 1 g C g ( g) h are depicted as: g C C D E g C D h The unit object I is represented by no wire ; or example ψ : I π : I π ψ : I I are depicted as: ψ π π oψ π ψ Morphisms ψ : I are called elements o. Operationally one can think o them as the states o system. 2. Compact closed categories. monoidal category is compact closed i or each object there are also objects r and l, and morphisms which satisy: 5 η l : I l η r : I r ɛ l : l I ɛ r : r I (1 ɛ l ) (η l 1 ) 1 (ɛ l 1 l) (1 l η l ) 1 l (ɛ r 1 ) (1 η r ) 1 (1 r ɛ r ) (η r 1 r) 1 r When depicting the morphisms η l, ɛ l, η r, ɛ r as l r l these axioms substantially simpliy to l i.e. they boil down to yanking wires. 5 These conditions guarantee that the category is closed, as explained in a simple diagrammatical manner in (Coecke 2007). r l r r l l r r g E C 3. Vector spaces, linear maps and tensor product. Let FVect be the category which has vector spaces over the base ield R as objects, linear maps as morphisms and the vector space tensor product as the monoidal tensor. To simpliy the presentation we assume that each vector space comes with an inner product, that is, it is an inner-product space. For the case o vector space models o meaning this is always the case, since we consider a ixed base, and a ixed base canonically induces an inner-product. The reader can veriy that compact closure arises, given a vector space V with base {e i } i, by setting V l V r V, and η l η r : R V V :: 1 i ɛ l ɛ r : V V R :: ij c ij ψ i φ j ij e i e i c ij ψ i φ j. So ɛ l ɛ r is the inner-product extended by linearity to the whole tensor product. Recall that i {e i } i is a base or V and i {e i } i is a base or W then {e i e j } ij is a base or V W. In the base {e i e j } ij or V V the linear map ɛ l ɛ r : V V R has as its matrix the row vector which has entry 1 or the base vectors e i e i and which has entry 0 or the base vectors e i e j with i j. The matrix o η l η r is the column vector obained by transposition. 4. Pregroups as compact closed categories. is an example o a posetal category, that is, a category which is also a poset. For a category this means that or any two objects there is either one or no morphism between them. In the case that this morphism is o type then we write, and in the case it is o type we write. The reader can then veriy that the axioms o a category guarantee that the relation on C is indeed a partial order. Conversely, any partially ordered set (P, ) is a category. For objects a, b P we take P (a, b) to be the singleton {a b} whenever a b, and empty otherwise. I a b and b c we deine a c to be the composite o the morphisms a b and b c. partially ordered monoid is a monoidal category with the monoid multiplication as tensor on objects; whenever a c and b d then we have a b c d by equation (1), and we deine this to be the tensor o morphisms a c and b d. iunctoriality, as well as any equational statement between morphisms in posetal categories, is trivially satisied, since there can only be one morphism between any two objects. Finally, each is a compact closed category or η l [1 p p l ] ɛ l [p l p 1] η r [1 p r p] ɛ r [p p r 1] and so the required equations are again trivially satisied. Diagrammatically, the links representing the type reductions in π (π r s o l ) o are exactly the caps :

5 r C l C o the compact closed structure. The symbolic counterpart o this diagrammatically depicted morphism is: ɛ r 1 ɛ l C : r C l C. Composing meanings quantiying type logic We have described two approaches to analysing structure and meaning in natural, one in which vector spaces are used to assign meanings to words in a, and another in which s are used to assign grammatical structure to sentences: vector space We aim or a theory that uniies both approaches, in which the compositional structure o s would lit to the level o assigning meaning to sentences and their constituents. Our approach proceeds as ollows: we look or a mathematical structure which comprises both the compositional structure and the vector space structure as ragments. some mathematical structure vector space ragment 1 ragment 2 Our previous mathematical analysis suggests the ollowing: objects vector space FVect cat. axioms This suggestion is problematic, however. The compact closed structure o FVect is somewhat degenerate since l r. Moreover, there are canonical isomorphisms V W W V which translate to posetal categories as a b b a. Thereore we have to reine types to retain the ull grammatical content obtained rom the analysis. There is an easy way o doing this: rather than objects in FVect we will consider objects in the product category FVect P, where P is the generated rom basic types. Explicitly, FVect is the category which has pairs (V, a) with V a vector space and a P as objects, and the ollowing pairs as morphisms: which we can also write as ( : V W, p q) (, ) : (V, p) (W, q). Note that i p q then there are no morphisms o type (V, p) (W, q). It is easy to veriy that the compact closed structure o FVect and P lits component-wise to one on FVect P : objects vector space FVect P cat. axioms We can now also lit the mechanism or establishing welltypedness o sentences within s to morphisms in FVect: given a reduction p q s, there is a morphism (, ) : (V, p)... (W, q) (S, s) which assigns to each vector v... w V... W (the meaning o a sentence o type p q) a vector: ( v... w ) S, where S is the meaning space o all sentences. Note that this resulting vector ( v... w ) does not depend on the grammatical type p q. Computing the meaning o a sentence We deine a meaning space to be a pair consisting o a vector space V and a grammatical type p, where, ollowing the vector space model o meaning, the vectors in V encode the meaning o words o type p. 6 To assign meaning to a sentence o type π(π r so l )o, we take the tensor product o the meaning spaces o its constituents, that is: (V, π) (T, π r so l ) (W, o) ( V T W, π(π r so l )o ). 6 The pair ( v, p), where v is a vector which represents the meaning o a word and p is the element which represents its type, is our counterpart o the pure tensor v p in the proposal o (Clark & Pulman 2007), in which p is a genuine vector. The structure proposed here can easily be adapted to also allow types to be represented in a vector space.

6 From the type π r so l o the transitive verb, we know that the vector space in which it is described is o the orm: The linear map which realizes T V S W. (2) ( V T W, o(π r so l )o ) (, ) (S, s), and arises rom the type-reductions, is in this case: ɛ r V 1 S ɛ l W : V T W S. Diagrammatically, the linear map can be represented as ollows: T V V T W The matrix o has dim(v ) 2 dim(s) dim(w ) 2 columns and dim(s) rows, and its entries are either 0 or 1. We can also express ( v Ψ w ) S or v Ψ w V S W in terms o the inner-product. I then Ψ ijk W c ijk v i s j w k V S W ( v Ψ w ) c ijk v v i s j w k w ijk ( ) c ik v v i w k w s j. j ik This vector is the meaning o the sentence o type π(π r so l )o, and assumes as given the meanings o its constituents v V, Ψ T and w W, obtained rom data using some suitable method. In Dirac notation, ( v Ψ w ) would be written as: ( ɛ r V 1 S ɛ r V ) v Ψ w. s mentioned in the introduction, our ocus in this paper is not on how to practically exploit the mathematical ramework, which would require substantial urther research, but to expose the mechanisms which govern it. To show that this particular computation does indeed produce a vector which captures the meaning o a sentence, we explicitly compute ( v Ψ w ) or some simple examples, with the intention o providing the reader with some insight into the underlying mechanisms and how the approach relates to existing rameworks. where we take V to be the vector space spanned by men and W the vector space spanned by women. 7 We will conveniently assume that all men are named John, using indices to distinguish them: John i. Similarly every woman will be reerred to as Mary j, or some j, and the set o vectors { Mary j } j spans W. Let us assume that John in (3) is John 3 and that Mary is Mary 4. ssume also that we are only interested in the truth or alsity o a sentence. Thereore we take the sentence space S to be spanned by a single vector 1, which we identiy with true, and we identiy the origin 0 with alse. The transitive verb likes is encoded as the superposition: likes ij John i likes ij Mary j where likes ij 1 i John i likes Mary j and likes ij 0 otherwise. O course, in practice, the vector that we have constructed here would be obtained automatically rom data using some suitable method. Finally, we obtain: ( John3 likes Mary ) 4 John3 John i likes ij Mary j Mary 4 ij δ 3i likesij δ j4 ij likes 34 So we indeed obtain the correct meaning o our sentence. Note in particular that the transitive verb acts as a unction requiring an argument o type subject on its let and another argument o type object on its right, in order to output an argument o type sentence. Example 2. We alter the above example as ollows. Let S be spanned by two vectors, l and h. Set loves John i loves ij Mary j ij hates ij John i hates ij Mary j where loves ij l i John i loves Mary j and loves ij 0 otherwise, and hates ij h i John i hates Mary j and hates ij 0 otherwise. Set likes 3 loves + 1 hates. 4 4 Example 1. Consider the sentence John likes Mary. (3) The reader can veriy that we obtain ( J 3 loves ) ( M 4 J 3 likes ) M We encode this sentence as ollows; we have: John V, likes T, Mary W 7 In terms o context vectors this means that each word is its own and only context vector, which is o course a ar too simple idealisation or practical purposes.

7 ( J 3 likes ) ( M 4 J 3 hates ) M ( J 3 loves ) ( M 4 J 3 hates ) M 4 0. Hence the meaning o the distinct verbs loves, likes and hates in the dierent sentences propagates through the reduction mechanism and reveals itsel when computing innerproducts between sentences in the sentence space. Due to lack o space we leave more involved examples to uture papers, including those which involve comparing the meanings o sentences o dierent types. O course in a ullblown vector space model, which has been automatically extracted rom large amounts o text, we obtain imperect vector representations or words, rather than the ideal ones presented here. ut the mechanism o how the meanings o words propagate to the meanings o sentences remains the same. Change o model while retaining the logic When ixing a base or each vector space we can think o FVect as a category o which the morphisms are matrices expressed in this base. These matrices have real numbers as entries. It turns out that i we consider matrices with entries not in (R, +, ), but in any other semiring 8 (R, +, ), we again obtain a compact closed category. This semiring does not have to be a ield, and can or example be the positive reals (R +, +, ), positive integers (N, +, ) or even ooleans (,, ). In the case o (,, ), we obtain an isomorphic copy o the category FRel o inite sets and relations with the cartesian product as tensor, as ollows. Let X be a set whose elements we have enumerated as X { x i 1 i X }. Each element can be seen as a column with a 1 at the row equal to its number and 0 in all other rows. Let Y { y j 1 j Y } be another enumerated set. relation r X Y is represented by an X Y matrix, where the entry in the ith column and jth row is 1 i (x i, y j ) r and 0 otherwise. The composite s r o relations r X Y and s Y Z is {(x, z) y Y : (x, y) r, (y, z) s}. The reader can veriy that this composition induces matrix multiplication o the corresponding matrices. Interestingly, in the world o relations (but not unctions) there is a notion o superposition (Coecke 2006). The relations o type r { } X (in matricial terms, all column vectors with 0 s and 1 s as entries) are in bijective correspondence with the subsets o X via the correspondence r {x X (, x) r}. Each such subset can be seen as the superposition o the elements it contains. The inner-product o two subsets is 0 i they are disjoint and 1 i they have a non-empty intersection. So we can think o two disjoint sets as being orthogonal. 8 semiring is a set together with two operations, addition and multiplication, or which we have a distributive law but no additive nor multiplicative inverses. Having an addition and multiplication o this kind suices to have a matrix calculus. Since the abstract nature o our procedure or assigning meaning to sentences did not depend on the particular choice o FVect we can now repeat it or the ollowing situation: powerset objects FRel P cat. axioms Montague-style semantics In FRel P we recover Montague semantics. Example 1 above was chosen in such a manner that it is essentially relational. The singleton { } has two subsets, namely { } and, which we respectively identiy with true and alse. We now have sets V, W and T V { } W with such that: V : {John i } i, likes T, W : {Mary j } j likes : {(John i,, Mary j ) John i likes Mary j } ij {John i } ij {Mary j } where ij is either { } or. Finally we have ({John 3 } likes {Mary 4 }) ({John 3 } {John i } ij ( {Mary j } {Mary 4 } ) ij 34. Future Work There are several implementation issues which need to be investigated. It would be good to have a Curry-Howard like isomorphism between non-commutative compact closed categories, bicompact linear logic (uszkowski 2001), and bramsky s planar lambda calculus. This will enable us to automatically obtain computations or the meaning and type assignments o our categorical setting. lso, Montaguestyle semantics lives in FRel P whereas truly distributed semantics lives in FVect P. It would be interesting to see where the so called non-logical axioms o Montague live and how they are maniested at the level o FVect P. Categorical axiomatics is lexible enough to accommodate mixed states (Selinger 2007), so in principle we can implement the proposals o (ruza & Widdows 2007).

8 cknowledgements ob Coecke is supported by EPSRC dvanced Research Fellowship EP/D072786/1 The Structure o Quantum Inormation and its Ramiications or IT and by EC-FP6- STREP Foundational structures or quantum inormation and computation. We thank Keith Van Rijsbergen and Stephen Pulman or their eedback at QI 07 in Stanord. Reerences bramsky, S., and Coecke, categorical semantics o quantum protocols. In Proceedings o the 19th nnual IEEE Symposium on Logic in Computer Science, IEEE Computer Science Press. arxiv:quantph/ bramsky, S., and Duncan, R. W categorical quantum logic. Mathematical Structures in Computer Science 16: argelli, D., and Lambek, J. 2001a. n algebraic approach to rabic sentence structure. Linguistic nalysis 31. argelli, D., and Lambek, J. 2001b. n algebraic approach to French sentence structure. Logical spects o Computational Linguistics. ruza, P., and Widdows, D Quantum inormation dynamics and open world science. I Press. Proceedings o I Spring Symposium on Quantum Interaction. uszkowski, W Lambek grammars based on s. Logical spects o Computational Linguistics. Casadio, C., and Lambek, J n algebraic analysis o clitic pronouns in Italian. Logical spects o Computational Linguistics. Clark, S., and Pulman, S Combining symbolic and distributional models o meaning. I Press. Proceedings o I Spring Symposium on Quantum Interaction. Coecke,. 2005a. Kindergarten quantum mechanics lecture notes. In Khrennikov,., ed., Quantum Theory: Reconsiderations o the Foundations III, IP Press. arxiv:quant-ph/ Coecke,. 2005b. Quantum inormation-low, concretely, and axiomatically. In Proceedings o Quantum Inormatics 2004, Proceedings o SPIE 5833, SPIE Publishing. arxiv:quant-ph/ Coecke, Introducing categories to the practicing physicist. In Sica, G., ed., What is category theory?, volume 30 o dvanced Studies in Mathematics and Logic. Polimetrica Publishing work/bob.coecke/cats.pd. Coecke, utomated quantum reasoning: Nonlogic semi-logic hyper-logic. I Press. Proceedings o I Spring Symposium on Quantum Interaction. Curran, J. R From Distributional to Semantic Similarity. Ph.D. Dissertation, University o Edinburgh. Dowty, D.; Wall, R.; and Peters, S Introduction to Montague Semantics. Dordrecht. Freyd, P., and Yetter, D raided compact closed categories with applications to low-dimensional topology. dvances in Mathematics 77: Gazdar, G Paradigm merger in natural processing. In Milner, R., and Wand, I., eds., Computing Tomorrow: Future Research Directions in Computer Science. Cambridge University Press Lambek, J The mathematics o sentence structure. merican Mathematics Monthly 65. Lambek, J Type grammar revisited. Logical spects o Computational Linguistics Lambek, J computational algebraic approach to English grammar. Syntax 7:2. Moortgat, M Categorical type logics. In van enthem, J., and ter Meulen,., eds., Handbook o Logic and Language. Elsevier. Preller, Towards discourse representation via grammars. JoLLI. Sadrzadeh, M Pregroup analysis o Persian sentences. In Casadio, C., and Lambek, J., eds., Recent computational algebraic approaches to morphology and syntax (to appear). ms6/perspregroup.pd. Sadrzadeh, M High-level quantum structures in linguistics and multi-agent systems. I Press. Proceedings o I Spring Symposium on Quantum Interaction. Schuetze, H utomatic word sense discrimination. Computational Linguistics 24(1): Selinger, P Dagger compact closed categories and completely positive maps. Electronic Notes in Theoretical Computer Science 170: Smolensky, P., and Legendre, G The Harmonic Mind: From Neural Computation to Optimality-Theoretic Grammar Vol. I: Cognitive rchitecture Vol. II: Linguistic and Philosophical Implications. MIT Press.