Overview Systematicity, connectionism vs symbolic AI, and universal properties COMP9844-2013s2 Systematicity divides cognitive capacities (human or machine) into equivalence classes. Standard example: a cognizer is not systematic with respect to subject/object in simple sentences, if it it can entertain the possibility (or understand) the sentence Mary but not the sentence Mary. So Mary and Mary are equivalent in the sense of being understandable by the same cognizer. No. Training a feedforward net on Mary doesn t mean it will recognise Mary. Mary train Mary test Mary?? Classical AI proponents forcefully pointed this out, back in the 1980s when systematicity was first considered. The connectionists, of course, fought back, tooth and nail. The problem is arbitrary choices of parameters (weights) depending on pseudo-random seed value, and not constrained away to insignificance by training data. Parameter values that produce a systematic system may be possible, but their choice is ad hoc.????
No though it was thought for years that they were...... but then Ken Aizawa, a philosopher, showed that they weren t. This doesn t mean that a classical system can t be systematic, just that classical does not imply systematic. The problem arises from the fact that arbitrary choices are made in designing classical systems - e.g. in Mary, a choice of grammar rules. S Person Person S Person S Person Mary Grammar rules that produce a systematic system may be possible, but their choice is ad hoc. 0 1 is a general mathematical theory of structure. It has been used in theoretical computer science since the 1970s Used in the creation of the Haskell programming language - functors and monads in Haskell are lifted straight from category theory. Functional programming research has also utilised the categorical concepts of catamorphisms, anamorphisms, and hylomorphisms. Definition of a Category A category has objects and arrows (aka morphisms). An arrow f goes between objects A, B. We write f : A B, or A f B A familiar example is the category Set. In Set, the objects are sets and the arrows are functions. There must be identity arrows - e.g. identity functions from a set to itself Arrows must be composable - e.g. f : A B and g : B C compose to form g f : A C - and this composition must be associative.
2 Category Theory 3: Commutative diagrams Commutative diagrams express some categorical ideas: Other examples of categories: semigroups and semigroup homomorphisms (Strings under concatenation form a semigroup: a semi-group homomorphism is a function f such that f (s 1 s 2 ) = f (s 1 ) f (s 2 ).) vector spaces and linear transformations (matrices) ditto groups, abelian groups, rings, fields, modules, metric spaces, topological spaces,... sets and relations (Rel) sets and partial functions (Par) a partially ordered set: objects are points a, b; there is a unique arrow a b if a b. (Morphisms are order-preserving functions.) A φ B f C The idea is that any two arrow composition paths between two objects in a commutative diagram are equal. So in this case commutativity means that g φ = θ f ( i.e. if A is a set, for all a A, g(φ(a)) = θ(f (a)). ) Another example - commutative if ρ = σ λ θ D g A λ B ρ C σ (1) (2) 1 is a categorical concept that can be used to show how to resolve the systematicity problem. The most general definition of universality is rather abstract. So we ll use an example of a universal property: categorical product In Set, categorical product is the familiar Cartesian product, together with the projection maps: π 1 : A B A : (a, b) a and π 2 : A B B : (a, b) b So the product is the triple (A B, π 1, π 2 ), though often it is written just as A B.
2 (A B, π 1, π 2 ) as above has this universal property: for any other triple (R, ρ 1, ρ 2 ) with ρ 1 : R A and ρ 2 : R B, there exists a unique function (arrow) u : R A B making the following diagram commute - i.e. π i u = ρ i for i = 1, 2: R ρ 1 ρ 2 u A A B B π 1 In essence, the product is the unique best object among triples of the form (R, ρ 1, ρ 2 ) with ρ 1 : R A and ρ 2 : R B - everything else factors through A B. This optimality, termed a universal property in category theory, is at the core of the category-theory-based theory of systematicity. π 2 (3) Tackling systematicity with products 1 Systematicity for symbolic AI vs NN enthusiasts Let s look at the problem of the systematicity of Mary. The general idea is that systematicity corresponds to a universality property, e.g. of a product. Then, as the product has its uniqueness property, no ad hoc-ness is involved. The reason that the grammar S Person Person is right is that, in it, S corresponds to a (non-trivial) product. Other forms of systematicity correspond to other universality properties. The task for symbolic AI folk is as follows: Find a suitable universality property for their instance of systematicity. Show that their symbolic AI approach is a model of this universality property. The task for NN folk is as follows: Find a suitable universality property for their instance of systematicity. Probably this is the same property as for the symbolic AI types. Show that their NN approach is a model of this universality property.
Systematicity would arise in nature (e.g. in biological brains) because the universality property makes for models that are efficient, hence favoured by evolutionary processes. provides the notion of universal properties. The notion of any A Z B factoring through the product A B can be seen as dividing a task into something like an interface module to deal with particulars of a particular instance of a problem, and a universal model that then performs/solves the task/problem. "Z" "A B" interface universal component Neither symbolic AI nor connectionism has a clear advantage here. References People Aizawa, K. The systematicity arguments. New York, Kluwer Academic (2003) Awodey S.. Oxford Logic Guides. New York: Oxford University Press (2006) Fodor, J., Pylyshyn, Z. Connectionism and cognitive architecture: A critical analysis. Cognition, 28 3-71 (1988) Phillips S. Are feedforward and recurrent networks systematic? analysis and implications for a connectionist cognitive architecture. Connection Science 10: 137160 (1998) Phillips, S., Wilson, W.H. Categorial compositionality: A category theory explanation for the systematicity of human cognition. PLoS Computational Biology 6(7), e1000858 (2010) Smolensky, P. The constituent structure of mental states: A reply to Fodor and Pylyshyn. The Southern Journal of Philosophy, 26 (Supplement) 137-161 (1987) Left: Saunders Mac Lane: defined categories (1945) Centre: Jerry Fodor: introduced the concept of systematicity (late 1980s) Right: Ken Aizawa: neither connectionism nor symbolic AI is necessarily systematic (2003)