Network Computing and State Space Semantics versus Symbol Manipulation and Sentential Representation

Transcription

1 Network Computing and State Space Semantics versus Symbol Manipulation and Sentential Representation Brain Unlike Turing Machine/Symbol Manipulator 1. Nervous system has massively parallel architecture different signals processed in millions of pathways simultaneously (Parallel Distributed Processing (PDP)) this makes for very fast response to certain kinds of complex computational problems (perceptual discrimination, motor coordination, speech recognition/production) see also 3. ( Could a Machine Think? p35-36 (56-59), M&C p154) 2. Brian s basic processing unit (neuron) is simple and analog ( Think? p35 (56)) 3. The hypothesis is that the brain represents in something like the way network systems are thought to represent. 4. Recurrent nature of many portions of the brain axons connecting one set of neurons to a second are matched by another set of axons projecting back to the first population. 5. Fault and damage tolerant cognitive processing is often not significantly affected by minor cell damage and scattered cell death, even when it gets to such a degree as to affect processing, processing quality degrades gradually as opposed to dropping off sharply this is, in part, a result of the parallel, distributed nature of the brain. Reliable even on degraded input 6. Brains are plastic transformational properties can be reconfigured this allows for change and learning and for certain parts of the brain to adapt to damage in other areas (fault tolerance again) 7. Not at all clear that cognitive processing is entirely or even mostly rule governed symbol manipulation. The physical structure suggests that it may be vector transformation. Network Systems Like Brains 1. Implement PDP 2. Processing unit simple, can be analog 3. Distributed representation and computation massively parallel structure means that i) long term representations are stored in a distributed manner across the network (the configuration of connection weights), this allows for quick access of relevant info; ii) short term representations are encoded in the activation vectors (activation levels or spiking frequencies) of a number (often very large) of neurons, and iii) transformations of activation vectors are carried out in parallel by a number (often very large) of units (M&C , Think? pp36-37 (56-59)) this related to 1. and 5. and A network can be recurrent, this i) allows it to modulate information processing in light of the immediate informational context, and ii) suggests that the brain is a complex dynamical system, the behavior of which is to some degree independent of its stimuli ( Think? p35 (56)) 5. The parallel, distributed nature of the computational and representational architecture in networks also exhibits these properties. ( Think? p37 (59) M&C p154) 6. Networks can also be plastic 7. The basic mode of operation is NOT rule governed symbol manipulation, but vector transformation 1

2 The claim is not that classical TM/SMs (or some slight modification) cannot have any of the above properties, but that networks (and, it is supposed, brains) naturally display these properties in virtue of their basic functional architecture, whereas an SM would have to be specially constructed and programmed to display these properties (also, a classical approach is going to have speed issues). Moreover, the Churchlands see the greatest prospect of generating artificial intelligence, and the greatest prospect of understanding terrestrial intelligence in general, in the empirical investigation of brains and network systems. Remember, however, that they make this not as an a priori claim, but as an empirical, and therefore falsifiable, claim. Some Key Concepts in Network Computation and Representation: Vector: A vector is just an ordered n-tuple of n quantities (e.g., <.2,.7,.5,.9>, is an ordered quadruple), which signifies the quantities of n (four) variables. A vector can be represented either as an n-tuple or as a point in an n- dimensional state-space. State-Space: an n-dimensional system of coordinates for representing information encoded as n-dimensional vectors. For instance, suppose we wanted a state space for encoding information about cars. We might start with a simple 1-dimensional space (a number line), and we could represent the model year of any given car as a point on that line. Next, suppose we want to include the Blue Book value of the car. We simply add a second dimension, giving us a 2-dimensional space (a Cartesian plane), and we encode the year and value as a point in that space which corresponds to the ordered pair of numbers for those quantities <2000, 17500>. (Note that the two scales need not be identical.) If we add average miles per gallon on the highway, we get a cube, and each car can be represented by a point in that cube space. If we add average MPG for city driving, we now have a 4-d solid (a hypercube here is where most of us start having difficulty visualizing, but that doesn t really matter). We can add dimensions for gross weight, safety ratings, reliability ratings, passenger capacity and cargo capacity, current mileage, repair and maintenance record, essentially whatever we think of. This is not a literal space, but a informational space it allows us to represent real properties of objects and their interrelations. Given that it is a geometrical mode of representation we can expect to find important and revealing relations of distance, proximity, and betweeness; important and revealing significance to certain sub-spaces (regions) within the overall state-space; or important and revealing significance to how a the position of a point or solid evolves over time (e.g., as age and mileage increase MPG ratings are likely to decrease, though less quickly for the more reliable cars with better maintenance and repair records...). Network: A system consisting of 2 or more layers of nodes, with each layer consisting of varying numbers of nodes. Individually, these nodes perform some simple (or in some cases more complex) arithmetical operations on their input and produce an output. The first layer, or input layer (usually at the bottom of the diagram), accepts a numerically encoded input an input vector; usually a series of quantities each of which ranges from 0 to 1, each node in the layer is fed a number. The final layer, or output layer, delivers a similar output vector (one number for each node in the output layer), which may then be interpreted by human users (Mine/Rock), become the input for other networks, or drive a motor system. The output layer nodes and those in any hidden layers (layers between the input and output) simply sum the weighted inputs from the nodes of the previous layer, the weighting of the input being a function of the connection weight between the nodes of the two layers. This results in a vector transformation (a change in the quantities and (possibly) the dimensionality of the vector), the basic form of computation in a network system. Each node at layer i > 1 then produces an output which is a non-linear function of its activation level (see NCP pp161, 172). If there are one or more hidden layers the varying connection weights between these are responsible for both the storage of long term representations, and the transformation of activation vectors. Activation Vector: An n-place vector representing the activation levels of n nodes in a network layer. Can be represented as an ordered n-tuple or as a point in a state-space (called an activation-vector space). Activation vectors can be interpreted as occurrent representations (e.g., of a particular taste, position of a limb, discrimination of an object) and regions of activation-vector space can be interpreted as prototype representations. Continuous occurrent representation (e.g., motion of a limb, motion of an object in a visual field, parsing of sentence structure) is understood as the motion of a point in activation space (as the activation levels in a layer evolve over time as a function of the changing input). 2

3 Connection Weights: Each node in layer i > 1 is input the value of each node from layer i - 1, and each layer i node sums these values. This would lead to each node in layer i having the same value (the sum of the layer i - 1 values), except that each input to an individual node in layer i is weighted by being multiplied by some number (the weight of that connection). (Also, weightings can be more complex functions, but we ll keep it simple.) The inputs from each layer i - 1 node receive different weightings as they arrive at different layer i nodes, and, as a result, any vector transformation can be computed. Each node at layer i then produces an output which is a non-linear function of its activation level (see NCP pp161, 172). Even a very simple three-layer network can compute complex vector transformations. The more nodes at each given layer, the higher the dimensionality of the activation vector and the more connections and weightings in the system, the more fine-grained the computational power. The connection weights can also be represented as a vector (a weight-vector), and so a particular configuration of weights for a layer in a network can be represented as a point in a high-dimensional state-space (a weight space). It is in the weight configuration that longer-term information is represented, for it is the weight vector which determines how the activation vectors are transformed. Learning, or training, is the adjustment of connection weights. Hence, change in the long term representation can be depicted as the evolution of a point through weight space. Training up: The process by which an artificial network is taught, or learns, or is trained for a particular task. This involves starting the network out with random connection-weightings, feeding the network input vectors for which the desired output vector is already known, and slightly adjusting the weightings in response to the degree of error in the actual output. (See NCP pp on the generalized delta rule.) Eventually, the weightings in the system settle into a configuration in which the degree of error is very small for the training set. The network can then produce reliable and correct output for the task on new sets of inputs. In some sense the network learns to discriminate mines from rocks, discriminate faces, colors, catch a ball, walk, parse a sentence, add numbers Example of vector transformation mathematics: Suppose we have a network like the one on the next page (M&C Fig. 7.17, OTC Fig. 5.3, NCP Fig. 9.4) with 4 nodes in the input layer, 3 in the hidden layer, and 4 in the output layer. Label the input nodes a, b, c, d. Label the 3 hidden layer nodes x, y, z. Each node (take x for example) has four weighted connections, one each from a, b, c, and d; label each of these connection weights x a, x b, x c, x d, and so for nodes y and z. Then the vector transformation yielding the activation vector for the hidden nodes x, y, z is calculated by matrix multiplication: < a, b, c, d > = xa xb xc xd < x, y y y y a b c d y, za z b z c zd z > this means multiply a times each of x a, y a, z a, b times x b, y b, z b, similarly for c and d, then sum the x columns to get the value for x, sum the y column and the z column, yielding the final (now 3-tuple) vector <x, y, z>. This is the activation vector and it is a straightforward mathematical function of the input vector < a, b, c, d> and the connection weights x a, y a, z a, x b, y b, z b, etc. A similar explanation appears with figure 5.8 from NCP. 3

4 4

5 5

6 6

7 7

8 8

9 9

10 Philosophy of Science: Sentential View of Theories Formalism of propositions and inference rules has immense difficulty accounting for rationality of theory development and testing o problems of induction (description & justification) o paradoxes of confirmation, indeterminacy of falsification o nature and rationality of laws Rationality of large scale conceptual change extremely problematic No good theory of learning (see center & right) Frame Problem (see center & right) Notion of simplicity seen as important, but its nature and a metric for measuring it left unexplained Poor understanding of perceptual knowledge and the theory laden nature of observation Kuhnian paradigms show how important non-linguistic elements are, but sentential view fails to address this in an integrated fashion such things as skill at experimental set-up and interpretation; variance in experimenters attitudes and responses to anomaly; the nature, difficulty, and rationality of conceptual change Nature of explanation problematic No contact with microstructure of brain, no explanation of how sentential representations implemented in thinkers Problems with Sentential Representation as Basic to Cognition Philosophy of Mind: FP account of Individual Representation and Inference Explanatory gaps between o human and non-human representation o representation in verbal and non-verbal children o sensory perception/imagination and thought as talking to oneself, including characterization of perceptual knowledge Learning in general poorly understood on the sentential paradigm Learning of a language must be explained in terms of the formulation and testing of hypotheses, but this either leads to regress or requires a pre-existing (innate) language plausibility of this highly problematic Difficulty of accounting for role of o background knowledge, esp. quick and relevant retrieval thereof; the Frame Problem (see right) o creative thought o context sensitivity o analogy, metaphor Difficulty accounting for nature of concepts o concepts as linguistically encoded necessary and sufficient conditions has various conceptual problems including: initial implausibility, implausibility of innateness requirement o prototype view of concepts more plausible, but understanding prototypes as linguistically encoded has difficulties: description of relevant prototype, similarity conditions and metric on similarity No contact with microstructure of brain Artificial Intelligence: Symbol Manipulator/Turing Machine approach Difficulty of dealing with background knowledge o storing a list of linguistically structured elements inefficient and implausible o the Frame Problem: ability to store and relevantly draw on background knowledge quickly and as needed very poor (exhaustive search: slow; relevant search: need to encode and search relevance conditions leads to regress and increase in storage costs Signal propagation in a computer a million times faster than in the brain, clock frequency of CPUs faster than any frequency in the brain, YET on many realistic tasks confronted by biological creatures (perception, motor control, speech recognition, etc.) the brain is far faster than a symbol manipulator Difficulty in accounting for learning and conceptual change (see center & left) No contact with microstructure of brain (see above )

11 State Space Semantics A View of Representation Based on Neurological and Network Computational Insights: Most basically, occurent representations can be understood as points, trajectories, volumes in the activation-vector space of a system; longer-term representations can be understood as points in the weight space. Transformations on representations ( inference, computation ) can be understood as vector transformations in a network system. The role of these representations and transformations is a result of the particular configuration of connection weights in the system, for the weight configuration partitions the activation-vector space into various sub-volumes and prototype regions. These are the longer-term representations. Ongoing activation follows a trajectory through that space. Since the configuration of weights can be represented as a point in the system s overall weight-space, we can consider the various points in weight space as various theories. Hence, learning and conceptual change can be accounted for as the evolution, over time, of the point through weight space i.e., the gradual adjustments of the system s connection weights, where this is carried out by some form of supervised or unsupervised learning process individual learning of the slow and structural kind Learning, especially after the initial connection weights are set in youth, can also be accounted for as the redeployment of conceptual resources some subset of prototypes and their internal structure can be redeployed to handle a new domain of phenomena. This is aided by the recurrent pathways in recurrent networks individual learning of the fast and dynamical kind Learning of language no longer needs to presuppose a pre-existing language, there is a more fundamental mode of representation and learning already in place Offers a more general and more fundamental mode of representation and transformation of representations, which may well apply across species and is able to ground other forms of representation, including symbolic and linguistically structured representation This does not produce any explanatory gaps, and indeed has the potential to show continuity where FP perceives gaps This suggests that the sentential view of theories and individual representation as well as inductive inference is highly superficial, no wonder it fails to explain our most complex cognition and most significant periods of (individual and social) conceptual change Nature of perceptual input to cognition clarified, no cognition without some particular weight configuration, all observation theory laden Background knowledge is not stored in list of sentences, and so the relevance search problem is avoided indeed, the relevant background knowledge is distributed across the network in the weightings, it does not need to be accessed, the vectors evolve over time simply in virtue of how the activation space is partitioned, so there is no problem, background info has an immediate and relevant effect on cognition/processing Context sensitivity is accounted for in a similar way, especially for recurrent networks those in which projections from some nodes/neurons reach back to connect to nodes earlier in the information flow. In such networks, recently processed information can have an immediate and ongoing modulating effect on incoming information. Holistic and Kuhnian insights seen to be consequences of the style of representation and cognition, including the nature and difficulty of paradigm shifts (escaping local error minima) the importance of situated knowledge (knowledge regarding one s bodily position and purposes in a spatio-temporal world) Yields powerful responses to/views of: problems of induction, natures and roles of explanation, inference to the best explanation, simplicity, analogy Massive parallel processing etc. makes contact with brain microstructure... see above...

12 Works Cited: NP Think? OTC M&C Churchland, Paul M. (1989) A Neurocomputational Perspective: The Nature of Mind and the Structure of Science. Cambridge, Mass.: The MIT Press. Churchland, Paul M. and Churchland, Patricia Smith. (1990) Could a Machine Think? Scientific American 262 (1): Churchland, Paul M. and Churchland, Patricia Smith. (1998) On the Contrary: Critical Essays, Cambridge, Mass.: The MIT Press. Churchland, Paul M. (1988) Matter and Consciousness: A Contemporary Introduction to the Philosophy of Mind. Revised Edition. Cambridge, Mass.: The MIT Press. 12