EE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, 2012 Sasidharan Sreedharan www.sasidharan.webs.com 3/1/2012 1
Syllabus Artificial Intelligence Systems- Neural Networks, fuzzy logic, genetic algorithms, Artificial neural networks: Biological neural networks, model of an artificial neuron, Activation functions, architectures, characteristics- learning methods, brief history of ANN research- Early ANN architectures (basics only)- McCulloh & Pitts model, Perceptron, ADALINE, MADALINE
Biological Neural Nets Pigeons as art experts (Watanabe et al. 1995) Experiment: Pigeon in Skinner box Present paintings of two different artists (e.g. Chagall / Van Gogh) Reward for pecking when presented a particular artist (e.g. Van Gogh)
Pigeons were able to discriminate between Van Gogh and Chagall with 95% accuracy (when presented with pictures they had been trained on) Discrimination still 85% successful for previously unseen paintings of the artists Pigeons do not simply memorise the pictures They can extract and recognise patterns (the style ) They generalise from the already seen to make predictions This is what neural networks (biological and artificial) are good at (unlike conventional computer)
Brain Computer: What is it? Human brain contains a massively interconnected net of 10 10-10 11 (10 billion) neurons (cortical cells) Biological Neuron - The simple arithmetic computing element 8
Biological Neurons 1. Soma or body cell - is a large, round central body in which almost all the logical functions of the neuron are realized. 2. The axon (output), is a nerve fibre attached to the soma which can serve as a final output channel of the neuron. An axon is usually highly branched. 3. The dendrites (inputs)- represent a highly branching tree of fibers. These long irregularly shaped nerve fibers (processes) are attached to the soma. 4. Synapses are specialized contacts on a neuron which are the termination points for the axons from other neurons. Synapses Dendrites Soma Axon from other neuron Axon Dendrite from other The schematic model of a biological neuron 9
Brain-like Computer Artificial Neural Network Mathematical Paradigms of Brain-Like Computer The new paradigm of computing mathematics Neurons and Neural Net consists of the combination of such artificial neurons into some artificial neuron net. Brain-Like Computer Brain-like computer is a mathematical model of humane-brain principles of computations. This computer consists of those elements which can be called the biological neuron prototypes, which are interconnected by direct links called connections and which cooperate to perform parallel distributed processing (PDP) in order to solve a desired computational task.? 10
ANN as a Brain-Like Computer NN as an model of brain-like Computer Brain The human brain is still not well understood and indeed its behavior is very complex! There are about 10 billion neurons in the human cortex and 60 trillion synapses of connections The brain is a highly complex, nonlinear and parallel computer (information-processing system) An artificial neural network (ANN) is a massively parallel distributed processor that has a natural property for storing experimental knowledge and making it available for use. It means that: Knowledge is acquired by the network through a learning (training) process; The strength of the interconnections between neurons is implemented by means of the synaptic weights used to store the knowledge. The learning process is a procedure of the adapting the weights with a learning algorithm in order to capture the knowledge. On more mathematically, the aim of the learning process is to map a given relation between inputs and output (outputs) 11 of the network.
Applications of Artificial Neural Networks Advance Robotics Intelligent Control Technical Diagnistics Machine Vision Artificial Intellect with Neural Networks Intelligent Data Analysis and Signal Processing Image & Pattern Recognition Intelligent Medicine Devices Intelligent Security Systems Intelligent Expert Systems 12
ANNs The basics ANNs incorporate the two fundamental components of biological neural nets: 1. Neurones (nodes) 2. Synapses (weights)
Neurone vs. Node
Structure of a node: Squashing function limits node output:
Synapse vs. weight
Feed-forward nets Information flow is unidirectional Data is presented to Input layer Passed on to Hidden Layer Passed on to Output layer Information is distributed Information processing is parallel Internal representation (interpretation) of data
Feeding data through the net: (1 0.25) + (0.5 (-1.5)) = 0.25 + (-0.75) = - 0.5 Squashing: 1 1 e 0.5 0.3775
Artificial neuron Human brain is a complex structure of highly interconnected network of simple processing element called neurons. The behaviour of a neuron can be captured as a simple model. Every component of the model bears a direct analogy to the actual constituents of a biological neuron and is termed as the artificial neuron and is the basis for ANN. Biological neuron receives all inputs through the dendrites sums them and produces an output if the sum is greater than a threshold value. The input signals are passed on to the cell body through the synapse which may accelerate and retard an arriving signal. The acceleration or retardation of the input signals are 19 modelled by the weights.
Artificial Neuron Simple Model I=w x +w x +...+w x n i=1 1 1 2 2 n n = w x i i y= (I):W here is activation function or transfer function or squash function which releases the output. 20
Artificial Neuron: Classical Activation Functions Linear activation Logistic activation I z 1 I 1 1 I e Σ input 0 input Threshold activation I 1, if I 0, sign ( I) 1, if I 0. 1 Hyperbolic tangent activation 1 e 1 e u tanh u 1 2 u 2 u -1 z - 1 0 21
1. Threshold Activation function y = w here n i 1 w x i is the step function know n as the H eaviside function ( I ) = 1, I 0 0, I 0 i Output signal is either 1 or 0 resulting in the neuron being on or off 22
2. Signum function ( I ) = 1, I 1, I 23
3. Sigmoid function The function is a continuous function that varies gradually between the asymptotic values 0 and 1 or -1 and +1 ( I ) 1 1 T e 24
4. Hyperbolic tangent function ( I) tanh( I) 25
McCulloch Pitts Model (1943)
Neural Network Architectures ANN is defined as a data processing system consisting of a large number of simple highly interconnected processing elements (artificial neurons). ANN structure can be represented by a directed graph (digraph directed graph or oriented graph) The vertices of the graph represents neurons (input/output) and the edges, the synaptic links. The edges are labelled by the weights attached to the synaptic links 27
1.Single Layer Feed Forward Network
2.Multi Layer Feed forward network Presence of hidden intermediate layer which are useful for computations.
3.Recurrent ANN Architecture Presence of at least one feedback loop. There could be self feedback links
Regards Dr. Sasidharan Sreedharan www.sasisreedhar.webs.com