Computational Intelligence Lecture : Fuzzy-Neural

Transcription

1 Lecture 1/34 Computational Intelligence Lecture : Fuzzy-Neural Arash Yeganeh Fallah Naser sepehry Department of Electrical Engineering Amirkabir University of Technology Fall 2010

2 Lecture 2/34 Basic Structure of Fuzzy Neural Networks Definition of Fuzzy Neurons Fuzzy Neural Networks Neural Network Representation of Fuzzy Relation Equations A Fuzzy Neural Network Based on FN( (V, Λ) A Fuzzy δ Learning Algorithm algorithm algorithm of FAM's(method 1) algorithm (method 2)

3 Lecture 3/34 Basic Structure of Fuzzy Neural Networks (FNN) Definition of Fuzzy Neurons The objective of FNN is to extend the capability of the neural networks to handle vague info ormation than crisp information only. Classifying of FNN: 1-a fuzzy neuron with crisp signalss used to evaluate fuzzy weights, 2-a fuzzy neuron with fuzzy signalss which is combined with fuzzy weights, 3-a fuzzy neuron described by fuzzy logic equations.

4 Lecture 4/34 A multifactorial function is actually a projective mapping from an m-array space to a one-array space, denoted by M m. a natural partial ordering is defined as follows Basic operators ( +,.) in multifactorialfunction change as bellow: Where V is MAX and Λis MIN

5 Lecture 5/34 A fuzzy neuron is regarded as a mapping FN : For This Figure illustrates the working mechanism of a fuzzy neuron.

6 Lecture 6/34 Example:

7 Fuzzy Neural Networks Neural Network Representation of Fuzzy Relation Equations A typical kind of fuzzy relation equation: Where X is input vector R is matrix of coefficients, B is constant matrix And operator o is defined as follow: Arash Yeganeh Fallah, Naser Sepehry Lecture 7/34

8 Lecture 8/34 network representing fuzzy relation equations: where the activation functions of the neurons f 1,f 2,.,f m are all taken as identity functions and the threshold values are zero Fuzzy relation equations can be solved using a fuzzy δlearning algorithm.

9 Lecture 9/34 A Fuzzy Neural Network Based on FN(V, Λ) The network is known to have fuzzy associative memories (FAM) ability based on FN(V, Λ). The relation between input and output of this network is as follows:

10 Lecture 10/34 Matrix form of FAM based on FN(V, Λ)is as follow For given a set of samples:

11 Lecture 11/34 Weight matrix Wby means of the following system of fuzzy relation equations can be obtained:

12 Lecture 12/34 Weight matrix Wby means of the following system of fuzzy relation equations can be obtained:

13 and, Weight matrix Wby means of the following system of fuzzy relation equations can be obtained: Arash Yeganeh Fallah, Naser Sepehry Lecture 13/34

14 Lecture 14/34 A Fuzzy δ Learning Algorithm Neural Network Representation of Fuzzy Relation Equations Describing procedures for the fuzzy δ learning algorithm:

15 Lecture 15/34

16 Lecture 16/34 Example: So,

17 Lecture 17/34 and

18 Lecture 18/34 several tests show that most values in Ware the same Except for w 31, w 33, and w 43. The following table details the difference.

19 Lecture 19/34 algorithm of FAM's The back propagation (BP) algorithm for the connection weight matrix W of the FAM is presented as follow

20 Lecture 20/34 Lemma 1: suppose d>1, s>0, and following estimation gets: x 1, x 2,, x d ϵ [0,1]. Then the Therefore,

21 Lecture 21/34 Lemma 2the functions La(s;x 1,,, x d ) and Sm(s;x 1,, x d )are continuously differentiable on [0,1]^d. moreover, for j ϵ {1,, d}, By Lemma 2, It can be concluded that the following facts hold for the constant a ϵ [0,1] :

22 So for a given constant a ϵ [0,1], it follows that algorithm Suppose {(x k, y k ) k ϵ P} is a fuzzy pattern pair family for training. The input pattern x k of FAM equations, the corresponding real output pattern will be: Arash Yeganeh Fallah, Naser Sepehry Lecture 22/34

23 Lecture 23/34 Where: The error function E(W) is definedd as follows: As E(W) is non-differentiable respect to w ij, thebp algorithm can not be designed by using E(W) directly. So the functions Laand Smis used to replaced the fuzzy operators V and Λ, respectively. By Lemma 1, when sis sufficiently large

24 Lecture 24/34 By Lemma 1, when sis sufficiently large Theorem 1 Give the fuzzy pattern pair family {(x k, y k ) k ϵ P}. Then e(w) is continuously differentiable with respect to w ij for iϵ N, j ϵ M Where

25 and Step of algorithm of FAM's: Step 1. Initialization. Put w ij (0) = 0, and let W(0) = (w ij (0)) n m, set t = 1. Step 2. Denote W(t) = (w ij (t)) n m. Step 3. Iteration scheme. W(t) iterates with the following law: Step 4. Stop condition. Discriminate ǀe(W(t+1))ǀ < ɛ? If yes, output w ij (t+1); otherwise, let t = t + 1 go to Step 2. Arash Yeganeh Fallah, Naser Sepehry Lecture 25/34

26 Lecture 26/34 Example:

27 Lecture 27/34 algorithm Modifications in weight space are performed according to: with learning error: Output layer: Hidden layer: Where learning rate ε:, momentumm α and offset β are constant values chosen by experience.

28 Lecture 28/34 Example for Standard Backpropagation: a set of 10 patterns (alphabetic letters from 'a' to 'j') has been presented for 0.1<ε<2.0 and 0<α< <1

29 Lecture 29/34 Fuzzy Adaption of Learning Parameters Fuzzification of learning error E(n) Fuzzification of error changes ΔE(n)

30 Lecture 30/34 Fuzzy set learning rate NB is restricted to and PB is restricted to 0.02 Fuzzy sets momentum NB and PB are restricted to and 0.02 Fuzzifiedconclusions (adaption of learning rate)

31 Lecture 31/34 Fuzzifiedconclusions (adaption of momentum) The center of gravity defuzzification method has been proposed

32 Lecture 32/34 Decision rule base for learning rate ε

33 Lecture 33/34 Decision rule base for momentumm α

34 Lecture 34/34 Standard and fuzzy control of learning rate and momentum Standard BP Fuzzy BP

35 Lecture 35/34 Puyin Liu, Hongxing Li, FUZZY NEURAL NETWORK THEORY AND APLICATION, World Scientific, 2004 Hongxing Li, C.L. Philip Chen, Han-Pang Huang, Fuzzy Neural Intelligent Systems, CRC Press, 2001 Gerke.M, Hoyer.H, Fuzzy BackpropagationTraining of Neural Networks, Lecture Notes in Computer Science, 1997, Volume 1226, Computational Intelligence Theory and Applications, Pages