Types in categorical linguistics (& elswhere)

Transcription

1 Types in categorical linguistics (& elswhere) Peter Hines Oxford Oct University of York N. V. M. S. Research

2 Topic of the talk: This talk will be about: Pure Category Theory.... although it might have interesting interpretations in various settings.

3 Possible interpretations: This pure category theory can be interpreted as: 1 Logic & theoretical computing. 2 Categorical linguistics / semantics. 3 Categorical quantum mechanics. Categorical linguistics will provide the fig-leaf for today s category theory.

4 Types in categorical models of meaning Distributional semantics: We construct meaning vectors for words Vectors can be compared, using an inner product. We derive a notion of distance between two words. Bringing in categorical linguistics: Moving to a typed system allows us to compare sentences.

5 A (oversimplified) description Words are assigned types, based on their rôle. This extends to sentences; these are typed by their grammatical structure. The types are objects in a monoidal closed category. closed categories have a reduction, or evaluation. All grammatical sentences reduce to the same type, S. By reducing sentences to the same type, they can be compared.

6 A reminder: A monoidal closed category C has: A monoidal tensor : C C C A unit object for the tensor I X = X = X I An internal hom [ ] : C op C C all satisfying various natural conditions. These can be used to define an evaluation arrow A [A B] eval A,B B

7 The kind of thing we wish to do... The aim: Use typing and evaluation, to reduce all sentences to the same type. Noun Intransitive Verb N [N S] N [N S] S Type individual words Combine types using the tensor Reduce, using the evaluation Types are chosen so all (well-formed) sentences reduce to S

8 The kind of thing we wish to do... The aim: Use typing and evaluation, to reduce all sentences to the same type. Noun Intransitive Verb N [N S] N [N S] S Type individual words Combine types using the tensor Reduce, using the evaluation Types are chosen so all (well-formed) sentences reduce to S

9 The kind of thing we wish to do... The aim: Use typing and evaluation, to reduce all sentences to the same type. Noun Intransitive Verb N [N S] N [N S] S Type individual words Combine types using the tensor Reduce, using the evaluation Types are chosen so all (well-formed) sentences reduce to S

10 The kind of thing we wish to do... The aim: Use typing and evaluation, to reduce all sentences to the same type. Noun Intransitive Verb N [N S] N [N S] S Type individual words Combine types using the tensor Reduce, using the evaluation Types are chosen so all (well-formed) sentences reduce to S

11 A few questions... How do we compare elements of the same type? How does comparison relate to 1 The monoidal tensor? 2 The internal hom.? 3 Evaluation? What does the sentence type S look like? Does evaluation lose information?

12 A few questions... How do we compare elements of the same type? How does comparison relate to 1 The monoidal tensor? 2 The internal hom.? 3 Evaluation? What does the sentence type S look like? Does evaluation lose information?

13 To avoid becoming too abstract(!) We will use & compare two example sentences L1. Bobby loves Marilyn Monroe. L2. I like Fidel Castro and his beard. These are both lyrics from Bob Dylan songs.

14 Defining elements of a certain type An element of a type T is an arrow from the unit object to T. Let the Noun Phrase type be N Ob(C). Then Bobby is an arrow I Bobby N Familiar examples: A member of a set is given by: a function f : { } X. A state in quantum mechanics is given by: a linear map ψ : C H.

15 Defining elements of a certain type An element of a type T is an arrow from the unit object to T. Let the Noun Phrase type be N Ob(C). Then Bobby is an arrow I Bobby N Familiar examples: A member of a set is given by: a function f : { } X. A state in quantum mechanics is given by: a linear map ψ : C H.

16 Defining elements of a certain type An element of a type T is an arrow from the unit object to T. Let the Noun Phrase type be N Ob(C). Then Bobby is an arrow I Bobby N Familiar examples: A member of a set is given by: a function f : { } X. A state in quantum mechanics is given by: a linear map ψ : C H.

17 Comparing elements (I) The precise form of categorical closure determines how we make comparisons. The grammar: Lambek pregroups form a (non-symmetric) compact closed category. The semantics Distributional semantics uses Vector Spaces another compact closed category. Tentative conclusion: let s use a compact closed category!

18 Comparing elements (I) The precise form of categorical closure determines how we make comparisons. The grammar: Lambek pregroups form a (non-symmetric) compact closed category. The semantics Distributional semantics uses Vector Spaces another compact closed category. Tentative conclusion: let s use a compact closed category!

19 Comparing elements (I) The precise form of categorical closure determines how we make comparisons. The grammar: Lambek pregroups form a (non-symmetric) compact closed category. The semantics Distributional semantics uses Vector Spaces another compact closed category. Tentative conclusion: let s use a compact closed category!

20 Comparing elements (II) CCCs are symmetric monoidal categories with duals ( ) f : A B duality f : B A The dagger ( ) is a contravariant (order-reversing) functor. In a CCC, the internal hom is [A B] = A B (This is a much simpler form that most closed categories).

21 Comparing elements(iii) Our examples have self-dual objects: A = A. Comparing elements of type N Elements Bobby and Fidel are compared using the composite: I Fidel N Bobby Bobby Fidel I The generalised scalar product is Bobby Fidel : I I.

22 Comparing elements(iii) Our examples have self-dual objects: A = A. Comparing elements of type N Elements Bobby and Fidel are compared using the composite: I Fidel N Bobby Bobby Fidel I The generalised scalar product is Bobby Fidel : I I.

23 Generalised scalar products Comparisons are of the form Bobby Fidel : I I In various categories, C(I, I) is: Real numbers R, complex numbers C, the unit interval [0, 1], the set {T, F}, the natural numbers N, etc. In general: A comparison u v gives a measure of the similarity or overlap of the elements u, v. For vector spaces, it is exactly the scalar product.

24 Some further points: The unit object I is not the sentence type S. For illustrative purposes, we will use C(I, I) = [0, 1] nothing in common exactly the same [0 1] Disclaimer: any actual values given are estimates (random guesses) x,y The comparison x y : I I exists for elements I A of the same type. this holds for any type A Ob(C).

25 Some further points: The unit object I is not the sentence type S. For illustrative purposes, we will use C(I, I) = [0, 1] nothing in common exactly the same [0 1] Disclaimer: any actual values given are estimates (random guesses) x,y The comparison x y : I I exists for elements I A of the same type. this holds for any type A Ob(C).

26 Some further points: The unit object I is not the sentence type S. For illustrative purposes, we will use C(I, I) = [0, 1] nothing in common exactly the same [0 1] Disclaimer: any actual values given are estimates (random guesses) x,y The comparison x y : I I exists for elements I A of the same type. this holds for any type A Ob(C).

27 Back to our sentences... L1. Bobby loves Marilyn Monroe. L2. I like Fidel Castro and his beard. Let s instantiate a variable... These are both Bob Dylan lyrics. We replace I by Bob Dylan.

28 Back to our sentences... L1. Bobby loves Marilyn Monroe. L2. Bob Dylan likes Fidel Castro and his beard. Let s instantiate a variable... We replace I by Bob Dylan,... and adjust the verb accordingly!

29 The first estimate... L1. Bobby loves Marilyn Monroe. L2. Bob Dylan likes Fidel Castro and his beard. Both Bobby and Bob Dylan are of type N we can form their scalar product. As a reasonable estimate (random guess?) we put Bobby Bob Dylan 0.98

30 Putting things in context From the context (i.e. Bob Dylan lyrics), we have assumed a close match between I and Bobby. Unfortunately... Historical / cultural context suggests that in L1. Bobby actually refers to Robert Kennedy However, this is not evident from the lyrics of either song.

31 Making more comparisons L1. Bobby loves Marilyn Monroe. L2. Bob Dylan likes Fidel Castro and his beard. These are both transitive verbs, so have type [N [N S]] As they have the same type, we may take their scalar product: likes loves 0.75 (Another random guess - from Mehrnoosh)

32 One last comparison... L1. Bobby loves Marilyn Monroe. L2. Bob Dylan likes Fidel Castro and his beard. How to compare Marilyn Monroe with Fidel Castro and his beard? These are em not the same type: Marilyn Monroe has type N Fidel Castro and his beard has type N C N where C is the type for a binary connective such as: AND, OR, EXCLUSIVE OR,...

33 One last comparison... L1. Bobby loves Marilyn Monroe. L2. Bob Dylan likes Fidel Castro and his beard. How to compare Marilyn Monroe with Fidel Castro and his beard? These are em not the same type: Marilyn Monroe has type N Fidel Castro and his beard has type N C N where C is the type for a binary connective such as: AND, OR, EXCLUSIVE OR,...

34 Typing connectives We wish for Fidel Castro and his beard N C N to reduce to something of type N. For this to happen, the connective type C must be [N [N N]]

35 We may now make a comparison: Applying an evaluation maps Fidel Castro and his beard into the type N. this can then be compared to Marilyn Monroe We are happy to guess (hope?) Marilyn Monroe Eval Castro and his beard = 0

36 A digression

37 A closer look at connectives The connective type C was chosen so that: N C N evaluates to N We wish for Noun Phrase and Noun Phrase to evaluate to another Noun Phrase. However, such connectives are used more generally.

38 Other contexts for connectives Bobby loves and obeys Marilyn Monroe Castro s big and bushy beard Bobby likes Marilyn and I like Fidel (Verb phrases) (Adjectives) (Entire sentences) The appropriate typing is: [X [X X]] where X varies, according to the context.

39 Other contexts for connectives Bobby loves and obeys Marilyn Monroe Castro s big and bushy beard Bobby likes Marilyn and I like Fidel (Verb phrases) (Adjectives) (Entire sentences) The appropriate typing is: [X [X X]] where X varies, according to the context.

40 Other contexts for connectives Bobby loves and obeys Marilyn Monroe Castro s big and bushy beard Bobby likes Marilyn and I like Fidel (Verb phrases) (Adjectives) (Entire sentences) The appropriate typing is: [X [X X]] where X varies, according to the context.

41 Other contexts for connectives Bobby loves and obeys Marilyn Monroe Castro s big and bushy beard Bobby likes Marilyn and I like Fidel (Verb phrases) (Adjectives) (Entire sentences) The appropriate typing is: [X [X X]] where X varies, according to the context.

42 Other contexts for connectives Bobby loves and obeys Marilyn Monroe Castro s big and bushy beard Bobby likes Marilyn and I like Fidel (Verb phrases) (Adjectives) (Entire sentences) The appropriate typing is: [X [X X]] where X varies, according to the context.

43 Connectives and polymorphism Our claim: To deal with connectives, we appear to need parameterised or polymorphic types. Abusing notation slightly, we write the type for and as ΛX. [X [X X]] or equivalently, ΛX. [X X X]

44 Connectives and polymorphism Our claim: To deal with connectives, we appear to need parameterised or polymorphic types. Abusing notation slightly, we write the type for and as ΛX. [X [X X]] or equivalently, ΛX. [X X X]

45 End of digression

46 Back to comparing sentences How does the scalar product interact with the tensor? Some simple category theory: Given scalar products a b : I I x y : I I the interaction with the tensor is simply: a x b y = a b. x y This is a general categorical identity.

47 Can we now compare our sentences? Using our (entirely fictitious) values: Bobby loves Marilyn Monroe. Bob Dylan likes Fidel Castro and his beard. Bobby Bob Dylan likes loves Fidel & his beard Marilyn Monroe

48 Can we now compare our sentences? Using our (entirely fictitious) values: Bobby loves Marilyn Monroe. Bob Dylan likes Fidel Castro and his beard. Bobby Bob Dylan likes loves Fidel & his beard Marilyn Monroe

49 Can we compare L1 and L2? We have two sentences, of type N [N [N S]] N We can take their inner product, to get L1 L2 = = 0 Important: We have compared L1 and L2 as elements of type N [N [N S]] N. Do we get the same answer if we first reduce them to terms of type S??

50 Can we compare L1 and L2? We have two sentences, of type N [N [N S]] N We can take their inner product, to get L1 L2 = = 0 Important: We have compared L1 and L2 as elements of type N [N [N S]] N. Do we get the same answer if we first reduce them to terms of type S??

51 Evaluation and scalar products Does evaluation preserve scalar products? x y I G x I Eval y S Is it true that x y? = x y

52 Does evaluation preserve scalar products? NO. The simplest counterexamples come from quantum mechanics, where evaluation is (partial) measurement. Evaluation is an irreversible operation A [A B] eval A,B B Is this desirable, or undesirable, for categorical models of meaning?

53 Does evaluation preserve scalar products? NO. The simplest counterexamples come from quantum mechanics, where evaluation is (partial) measurement. Evaluation is an irreversible operation A [A B] eval A,B B Is this desirable, or undesirable, for categorical models of meaning?

54 Some motivation: Scruffy Cats In distributional semantics: The element I Cat N provides information about cats in general... An element I Scruffy [N N] might tell us about the general concept of scruffiness. The tensor product I Scruffy Cat [N N] N tells us all about scruffiness, along with everything about cats.

55 Some motivation: Scruffy Cats In distributional semantics: The element I Cat N provides information about cats in general... An element I Scruffy [N N] might tell us about the general concept of scruffiness. The tensor product I Scruffy Cat [N N] N tells us all about scruffiness, along with everything about cats.

56 Evaluation, and forgetfulness The element provides too much information! I Scruffy Cat [N N] N Composing with the evaluation map: I Scruffy Cat [N N] N Eval N defines a new element, that tells us about Scruffy Cats only. It is vital that evaluation can forget information.

57 Evaluation, and forgetfulness The element provides too much information! I Scruffy Cat [N N] N Composing with the evaluation map: I Scruffy Cat [N N] N Eval N defines a new element, that tells us about Scruffy Cats only. It is vital that evaluation can forget information.

58 A more structural point of view Taking a logical view of our type system: We work with compact closure This corresponds to a (degenerate) fragment of Linear Logic. This is resource-sensitive. (For example) the resource I Scruffy [N N] is consumed in the evaluation... and plays no further rôle.

59 How about a limited form of reversibility? Let us compare Cat : I N Dog : I N do we get the same value when we compare Eval (Scruffy Cat) : I N, Eval (Scruffy Dog) : I N? In general, no!

60 How about a limited form of reversibility? Let us compare Cat : I N Dog : I N do we get the same value when we compare Eval (Scruffy Cat) : I N, Eval (Scruffy Dog) : I N? In general, no!

61 In closed categories Elements C(I, [X Y ]) are in 1:1 correspondence with Arrows C(X, Y ) Most elements do not correspond to isomorphisms!

62 A special case: In Hilb FD The element C Ψ H K = [H K ] maps to the arrow H L Ψ K L Ψ : H K is unitary exactly when Ψ is maximally entangled! This is, of course, a very special condition.

63 Must evaluation always lose information? Sometimes, it is undesirable for reduction to lose information! An example... Fidel Castro and his beard N [N N N] N N The compound noun-phrase The typing After evaluation The arrow named by I and [N N N] should not lose information about either 1 Fidel Castro, 2 Fidel Castro s beard.

64 Must evaluation always lose information? Sometimes, it is undesirable for reduction to lose information! An example... Fidel Castro and his beard N [N N N] N N The compound noun-phrase The typing After evaluation The arrow named by I and [N N N] should not lose information about either 1 Fidel Castro, 2 Fidel Castro s beard.

65 A more serious example The (polymorphic) connective type ΛX.[X X X] can be applied to the sentence type S Bobby likes Marilyn Monroe and I like Fidel Castro We do not wish the evaluation S [S S S] S Eval S to lose information about either sub-sentence.

66 Polymorphism and reversibility The arrow S S S named by I and [S S S] must be a monomorphism. This is closely related to models of polymorphic types.

67 building polymorphic types a special case We require an embedding: C(S S, S S) C(S, S) S contains a copy of S S A special case We look at the special case where this is an isomorphism: 1 S S S S S 1 S = 1 S, = 1 S S The two situations are (broadly speaking) interchangeable.

68 building polymorphic types a special case We require an embedding: C(S S, S S) C(S, S) S contains a copy of S S A special case We look at the special case where this is an isomorphism: 1 S S S S S 1 S = 1 S, = 1 S S The two situations are (broadly speaking) interchangeable.

69 A distinguished, closed, subcategory Consider the subcategory of C generated by S Ob(C), ( ) We have the following isomorphisms: S S = S [S S] = S S = S S = S We have a compact closed subcategory 1 where all objects are isomorphic. 1 without unit object

70 A distinguished, closed, subcategory Consider the subcategory of C generated by S Ob(C), ( ) We have the following isomorphisms: S S = S [S S] = S S = S S = S We have a compact closed subcategory 1 where all objects are isomorphic. 1 without unit object

71 A distinguished, closed, subcategory Consider the subcategory of C generated by S Ob(C), ( ) We have the following isomorphisms: S S = S [S S] = S S = S S = S We have a compact closed subcategory 1 where all objects are isomorphic. 1 without unit object

72 IMPORTANT! In this subcategory, we cannot assume strict associativity A (B C) = (A B) C Associativity must be up to canonical isomorphism: t ABC : A (B C) (A B) C A classic result (J. Isbell / S. MacLane) Trying to combine: Strict associativity S (S S) = (S S) S self-similarity S = S S forces S to collapse to the unit object. Categories for the working mathematician uses this to justify associativity up to isomorphism instead of strict associativity.

73 IMPORTANT! In this subcategory, we cannot assume strict associativity A (B C) = (A B) C Associativity must be up to canonical isomorphism: t ABC : A (B C) (A B) C A classic result (J. Isbell / S. MacLane) Trying to combine: Strict associativity S (S S) = (S S) S self-similarity S = S S forces S to collapse to the unit object. Categories for the working mathematician uses this to justify associativity up to isomorphism instead of strict associativity.

74 Another digression (for logicians & hardcore category-theorists)

75 Compact closed monoids The identities S = S S = [S S] look like the defining equations of a C-monoid (a Cartesian closed monoid / model of untyped λ- calculus). This analogy can be taken seriously For any object X of this subcategory, C(X, X) is a compact closed monoid.

76 The structure of C(S, S) This has a monoidal tensor : C(S, S) C(S, S) C(S, S) This is defined by convolution: S S S f g S f g S S This is: Associative (up to isomorphism) Commutative (up to isomorphism) However, there is no unit object.

77 The structure of C(S, S) This has a monoidal tensor : C(S, S) C(S, S) C(S, S) This is defined by convolution: S S S f g S f g S S This is: Associative (up to isomorphism) Commutative (up to isomorphism) However, there is no unit object.

78 Associativity of There is an associativity isomorphism t C(S, S) satisfying: t.(f (g h)) = ((f g) h).t MacLane s pentagon condition. It also satisfies: S S S 1 S S (S S) t S S S 1 S t S,S,S (S S) S

79 Associativity of There is an associativity isomorphism t C(S, S) satisfying: t.(f (g h)) = ((f g) h).t MacLane s pentagon condition. It also satisfies: S S S 1 S S (S S) t S S S 1 S t S,S,S (S S) S

80 Symmetry of There is also a symmetry isomorphism σ C(S, S) satisfying σ.(f g) = (g f ).σ MacLane s hexagon condition. It also satisfies: S S S σ S σ S,S S S

81 Symmetry of There is also a symmetry isomorphism σ C(S, S) satisfying σ.(f g) = (g f ).σ MacLane s hexagon condition. It also satisfies: S S S σ S σ S,S S S

82 End of digression

83 Back on track... Building models of polymorphism depends on: A distinguished object S Ob(C). Distinguished isomorphisms: : S S S : S S S. We also assume = = 1 this hold in most concrete examples! Question: do we have a Frobenius algebra?

84 Ceci n est pas un Frobenius algebra This fails at the first step: Units are a problem There are no natural candidates for the units : I S, : S I How about a Frobenius algebra without units?

85 What about associativity? In a Frobenius algebra, we need associativity S S S S S S ( 1 S ) : S (S S) S

86 What about associativity? In a Frobenius algebra, we need associativity S S S S S S ( 1 S ) : (S S) S S

87 (Strict) Associativity fails! The (strict) associative condition for a Frobenius algebra fails... for deeply unsatisfactory reasons! We do have associativity up to isomorphism.

88 We have associativity, up to isomorphism Adding in canonical isomorphisms: S S S 1 S S (S S) t t S,S,S S S S 1S (S S) S (Recall t C(S, S), the associativity arrow for )

89 We have associativity, up to isomorphism The same canonical isomorphisms make the dual diagram commute: S (S S) 1 S S S S t S,S,S t (S S) S 1S S S S We have associativity, and co-associativity, up to isomorphism.

90 We have lax monoids / comonoids Provided we don t care about units: We have a (lax) monoid and comonoid at S. We call these unitless monoids / comonoids, even though a monoid without a unit is a semigroup How about the Frobenius condition?

91 We have lax monoids / comonoids Provided we don t care about units: We have a (lax) monoid and comonoid at S. We call these unitless monoids / comonoids, even though a monoid without a unit is a semigroup How about the Frobenius condition?

92 We have lax monoids / comonoids Provided we don t care about units: We have a (lax) monoid and comonoid at S. We call these unitless monoids / comonoids, even though a monoid without a unit is a semigroup How about the Frobenius condition?

93 The Frobenius condition? The Frobenius condition requires: The composite: S S S S S S

94 The Frobenius condition? The Frobenius condition requires: is equal to S S S S

95 As a commutative diagram The Frobenius condition S S S S 1 S 1 S S S S S S S Strict equality! We replace strict associativity by isomorphism:

96 The Frobenius condition (up to iso.) The following is satisfied: S S 1 S (S S) S t 1 t 1 S,S,S S S S (S S) 1 S We have the Frobenius condition, up to canonical isomorphism.

97 Anything else? We have a unitless Frobenius algebra (up to canonical iso.) anything else?? We have commutativity & co-commutativity e.g. S S S σ S,S S S Again, up to canonical isomorphism. S σ

98 Anything else? We have a unitless Frobenius algebra (up to canonical iso.) anything else?? We have commutativity & co-commutativity e.g. S S S σ S,S S S Again, up to canonical isomorphism. S σ

99 One final point We also have the classical structure condition: (This was our starting point!) = 1 S Conclusion: the polymorphism condition, S = S S, leads to a (lax, unitless) classical structure as used to specify orthonormal bases in categorical quantum mechanics.

100 One final point We also have the classical structure condition: (This was our starting point!) = 1 S Conclusion: the polymorphism condition, S = S S, leads to a (lax, unitless) classical structure as used to specify orthonormal bases in categorical quantum mechanics.

101 The real conclusion: I had to say something, to strike them kind of weird, so I yelled I like Fidel Castro, and his beard. Bob Dylan, Motorpsycho Nightmare A similar result can be obtained by talking about polymorphism.