FEEDFORWARD NETWORKS WITH FUZZY SIGNALS 1

Transcription

1 FEEDFORWARD NETWORKS WITH FUZZY SIGNALS R. Bělohlávek Institute for Research and Applications of Fuzzy Modeling University of Ostrava Bráfova Ostrava Czech Republic belohlav@osu.cz and Department of Computer Science Technical University of Ostrava tř. 7. listopadu Ostrava-Poruba Czech Republic Abstract. The paper discusses feedforard neural netorks ith fuzzy signals. We analyze the feedforard phase and sho some properties of the output function. Then e present a backpropagation like adaptation algorithm for crisp eights thresholds and neuron slopes of the multilayer netork ith sigmoidal transfer functions. We provide theoretical ustification for the adaptation formulas. The results are of general nature and together ith the presented approach can be used for other types of feedforard netorks. Proposed and discussed are also applications of the presented feedforard netorks. Key ords: feedforard neural netorks fuzzy sets extension principle adaptation backpropagation. Introduction Fuzzy methods and neural netorks constitute the essential part of soft-computing a discipline aiming at methods capable of dealing ith complex humanistic systems. Among the most important features of neural netorks e recognize first of all the massively parallel architecture and dynamics hich is inspired by the structure of human brain the adaptation capabilities and the fault tolerance. On the other hand fuzzy sets represent an useful model of vagueness a phenomenon playing crucial role in the ay humans regard the orld. The significant properties of fuzzy methods are the efficient processing of the modeled indeterminacy especially the capability of modeling and further processing of natural language semantics. This results in human-friendly models the models inputs and outputs can be expressed or interpreted on the level of natural language or have other clearly understandable meaning. Both neural netorks and fuzzy methods are closely related ith human cognition hoever on different levels. Neural netorks concern the microlevel the level of brain. Fuzzy methods concern the macrolevel the level of mental phenomena. The levels (micro and macro) are strongly connected. The macrolevel seems to be implemented in the microlevel. The analysis of this combination of the levels is fundamental for our understanding of human cognition. The study of the analogous combination of the soft-computing-methodics counterparts of the levels i.e. neural netorks and fuzzy methods is therefore orth pursuing. Not surprisingly neural and fuzzy models can be combined in many ays. The resulting models can be built up of separate neural and fuzzy modules performing relatively independent tasks they can be based on neural models supplied by some features of fuzzy models (esp. processing of indeterminate information) or conversely based on fuzzy models supplied by features of neural Supported by grant No. 20/96/0985 of the GAČR and partially by IGS of University of Ostrava No. 03/97 029/98.

2 netorks (esp. adaptability). An overvie of various kinds of these models called neuro-fuzzy or fuzzy-neuro models can be found e.g. in [4 7 6]. The combinations of neuro and fuzzy models presented in literature are often ad hoc approaches ithout correct mathematical treatment. To bring some order into the development it is necessary to delineate and to pursue some paradigms of the combination. A useful paradigm is represented by Zadeh s extension principle (EP) [9 2 4]. The combination based on EP has been firstly presented probably in [4]. Our paper deals ith this approach and is organized as follos. In Section 2 e propose the architecture and deal ith the feedforard phase of our model. In Section 3 e propose an adaptation procedure and derive the adaptation formulas. Possible applications are discussed in Section 4. An example demonstrating the learning and interpolation capabilities is presented in Section 5. Section 6 briefly summarizes the results. 2 Architecture and feedforard phase By a feedforard (neural) netork [] it is usually understood a netorked computational scheme composed of several simple computational units hich are connected in such a ay that the graph of information flo contains no loops (the information is fed forard). The simple units process information in a ay analogous to that of biological neurons. The adective neural is due to the analogy ith biological neural netorks. The feedforard netorks take their inputs process them in the feedforard phase and return the outputs. The inputs to the netork are usually real numbers. In hat follos e discuss the case here the inputs and the outputs are not real numbers but fuzzy sets in the real line. The reason and advantages of having fuzzy sets instead of crisp real numbers are discussed in Section 4. There are several particular types of feedforard netorks see e.g. []. Probably the most popular one is the multilayer feedforard netork ith sigmoidal transfer functions called also backpropagation net because of its adaptation procedure [7]. This netork ill be a subect of our investigation. Hoever our results hich concern the general case of feedforard netorks ill be formulated as general as necessary for application to other types of feedforard netorks. We no briefly describe the multilayer feedforard neural net. We suppose the th neuron of the l th layer to have the sigmoidal transfer function y (l) y (l). The input z(l) layer i.e. z (l) m l i (l) +e λ(l) z(l) here λ (l) is the slope of to the neuron is the thresholded eighted sum of the outputs of the preceding i y(l ) i θ (l). Here (l) i denotes the eight of the connection leading from is the threshold. the i th neuron of the (l ) th layer to the th neuron of the l th layer and θ (l) A part of the netork is depicted in Fig.. Let m l and r denote the number of neurons in the l th layer and the number of layers in the netork respectively. The inputs to the netork are formally the outputs of the 0 th layer. Such a netork performs a mapping F : R m 0 (0) mr. A suitable transformation can guarantee that the outputs are from R mr. From no on e suppose the inputs to the neural netork to be fuzzy sets in R instead of real numbers. One can then use the EP to extend the operations performed by the neural netork to fuzzy sets. We apply the EP to each operation performed by the net. To be able to cope effectively ith EP e restrict ourselves to the class F CI (R) of normal convex fuzzy sets in R ith compact α cuts for α (0]. Recall that a fuzzy set A is normal if there is an x ith A(x) ; the α cut of A is the set α A x;a(x) α}; A is convex if each of its α cuts is convex i.e. an interval in our case; each fuzzy set x may be represented by a collection α x;α (0]} of its α-cuts. Under 2

3 y (l) θ (l) λ(l) z (l) (l) i y (l ) i i Figure : Part of the underlying neural netork. our conditions for a continuous f : R n R e have for f obtained by EP α f(x...x n ) f( α x... α x n ) for x i F CI (R) i.e. the operations on fuzzy sets can be performed by interval operations see e.g. [2]. Furthermore if f is monotone the interval operations are easy to compute. We no describe the feedforard phase. The netork dynamics is the classical one of the feedforard neural netork. After a tupple of signals is presented the netork transfers signals layer by layer. For x i F CI (R) e denote α x i [ α x Li α x Ri ]. Suppose e have the output interval signals y (l ) i F CI (R) of all the neurons of the (l ) th layer at disposal. As mentioned above e may represent y (l ) i by a collection of intervals [ α y (l ) Li α y (l ) Ri ]. The total input to the th neuron in the l-th layer is then given by m l [ α z (l) L α z (l) R ] hich leads by the interval arithmetic to i (l) i [α y (l ) Li α y (l ) Ri ] θ (l) [] and α z (l) L α z (l) R ml i (l) i 0 m l i (l) i 0 (l) i α y (l ) Li + (l) i α y (l ) Ri + m l i (l) i <0 m l i (l) i <0 (l) i α y (l ) Ri θ (l) (l) i α y (l ) Li θ (l) (). (2) As the transfer function y(z) that for λ (l) +e λ(l) 0 y(z) 2 ) e have is increasing for λ(l) z > 0 and decreasing for λ (l) < 0 (note and [ α y (l) L α y (l) R ] [ [ α y (l) L α y (l) R ] [ + e λ(l) α z (l) L + e λ(l) α z (l) R + e λ(l) α z (l) R + e λ(l) α z (l) L ] ] for λ (l) 0 for λ (l) < 0. 3

4 The described signal transfer is performed successively for the layers l... r. Return no to our problem from the functional point of vie. For a given (regular) neural netork hich represents a mapping F : R m 0 (0) mr e have obtained a netork ith fuzzy signals hich represents a mapping F : F CI (R) m 0 F CI ((0)) mr. A natural question at this point is hat is the relationship of F to the function F obtained from F by the EP. We ill anser this question in general. We face the task of extending the function F hich is in a certain ay composed of several simple functions. Extension by EP can be done in to ays extend the hole composite function (in our case to F) or extend the simple constituent functions and compose them (in our case to F). In the to folloing statements e suppose that the structure of truth values forms a complete lattice of hich the usual interval [0] is a special case [4]. We need the folloing lemma. Lemma Let L L be a complete lattice K i L i I K L such that K i ; i I} K. Then it holds Ki ; i I} K. Proof Denote a K a i K i. We have to prove a i ; i I} a. K i K implies a i a for all i I. Hence a i ; i I} a. Conversely for each b K there is a such that b K and thus b a a i ; i I}. Hence a K b K} a i ; i I}. We have ai ; i I} a. The folloing theorem shos the relationship of the to ays of extending a composite function. Theorem 2 Let g : Y m Z f i : X k Y i... m be functions A...A k fuzzy sets in X. Then for the corresponding extensions it holds If moreover m then even holds. g(f... f m )(A... A k ) g(f...f m )(A... A k ). Proof By the extension principle e have and g(f )(A... A k ) g(f )(A... A k ) g(f... f m )(A...A k )(z) g(f (A...A k )... f m (A... A k ))(z) g(f (A... A k ) f m (A...A k ))(z) g(f (A A k ) f m (A A k ))(z) (f (A A k ) f m (A A k ))(y... y m ); g(y... y m ) z} m A (x ) A k (x k );f i (x... x k ) y i }}; g(y... y m ) z} a i g(f...f m )(A... A k )(z) g(f... f m )(A A k )(z) A (x ) A k (x k ); g(f...f m )(x... x k ) z} b 4

5 Denote K A (x ) A k (x k ); g(f... f m )(x... x k ) z} K yi A (x ) A k (x k ); y y... y i... y m g (z) f i (x...x k ) y i y}. We have a ni z g (y) yi K yi and b K. Clearly K K yi for each y and i. By ellknon lattice properties it follos K yi K yi hence K n i yi K yi hence b K ni y g (z) yi K yi a hich proves the first part. For m the assertion follos immediately from Lemma putting K i K y. From Theorem 2 e get by induction over the number of layers the folloing statement. Corollary 3 Let x... x m0 be fuzzy sets in R and F(x... x m0 ) y... y mr F(x... x m0 ) ŷ...ŷ mr here F and F are the functions described above. Then for i... m r it holds y i ŷ i. It can be easily demonstrated [3] that the equality of the output fuzzy sets does not hold in general even in the case of our netorks. In other ords the output fuzzy sets of F are ider that the output fuzzy sets of F. Hoever for crisp numbers taken as singletons the functions F F and F coincide hich has a positive consequence: If the inputs have one-element kernels (hich is often the case) then the kernels of the outputs of F and F coincide and are also one-element. We further progress ith the adaptation of our netork. Given a training set T X p O p x... x n o...o m F CI (R) n F CI ((0)) m p P } e ant to design a netork ith m 0 n m r m and adapt it to T. Adaptation in the case of feedforard netorks means the design of a netork hich approximately represents the mapping partially prescribed by T. Hence e ant F to satisfy F(X p ) O p for all p P here is some further unspecified relation approximately equals. We omit the phase of determining the number of layers and neurons in layers and suppose this has been done. Our task is to find appropriate parameters. We search not only for eights but also for thresholds and slopes see [3]. Adaptation leads to the minimization process of an appropriate error function for hich e choose the function here E p P C β(α k ) α k E p k α E p α E p L + α E p R 2 mr i (α y (r) Li α o p Li )2 + 2 mr i (α y (r) Ri α o p Ri )2 and α α C β(α ) β(α C ). The β(α k ) is the eight of importance of the α k -cut C represents the discretization. In order to minimize the error function one usually uses the method of the steepest descent approach. Recall that by the direction of the steepest descent it is meant the direction in hich the function falls most rapidly by infinitesimal changes in this direction. We search in R W+2N W is the number of eights N is the number of neurons in the net for the point p... (l) i...θ(l)...λ(l)... in hich E takes the minimum. By the standard method of the 5

6 steepest descent one chooses an initial p(0) e.g. at random and then at any time t determines the change p(t) p(t) ηd(p(t)) + µ p(t ) here d(p(t)) is the direction of the steepest descent of E in the point p(t) η > 0 is the length of the step (learning rate) and µ is a momentum constant. We have naturally arrived at the point here interval netorks i.e. netorks that ork ith real intervals instead of real numbers see [ ] can be exploited. From () and (2) it is clear that the functions α E p (and hence E) are not differentiable in general. The author in [8] uses a little trick to get differentiable error function. Formulas () and (2) can be reritten as follos and ml α z (l) L i (sgn + ( (l) i )(l) i α y (l ) Li + sgn ( (l) i )(l) i α y (l ) Ri ) θ (l) m l α z (l) L i (sgn ( (l) i )(l) i α y (l ) Li + sgn + ( (l) i )(l) i α y (l ) Ri ) θ (l) here sgn + (x) is the signum function and sgn sgn +. The undifferentiable functions sgn + and sgn are in [8] substituted by their differentiable analogies s + and s respectively here s + (x) + e x and s (x) + e x. The function E is then differentiable hoever this approximation leads to an inaccurate interval arithmetic. The main disadvantage of this modified architecture hich invalidates it for use for our purposes in netorks ith fuzzy signals is the subsethood nonmonotonicity see [2]. We say that function mapping n-tupples of intervals (or fuzzy sets) to m-tupples of intervals (or fuzzy sets) is subsethood monotonic if for every interval inputs x i x i i... n ith x i x i e have y y... m for the corresponding outputs. The folloing proposition can be easily proved. Proposition 4 Let f be a function obtained from f : R n R by the Extension Principle. Then for every fuzzy sets x i x i in R i... n such that x i x i e have f(x...x n ) f(x...x n ). By induction it follos that the mapping F is subsethood monotonic. As a special case e obtain the subsethood monotonicity for intervals. It follos that the feedforard netork ith fuzzy signals has to be subsethood monotonic i.e. the netork ith approximate sgn + sgn is not satisfactory. The practical result of using a nonmonotonic netork could be that from an α cut representation of input fuzzy sets one gets a system of α cuts hich do not represent any output fuzzy sets at all (it could happen α F(x) β F(x) for β < α and m 0 m r ). 3 Adaptation We no sho that a backpropagation like steepest descent algorithm is possible and ell ustified. We need the folloing concepts. By a signature e mean a tupple Σ Σ... Σ n here Σ i + ±} for i... n. If Σ is a signature then by R Σ e denote the set R Σ x... x n R n ; x i 0 for Σ i + x i 0 for Σ i x i R for Σ i ± i... n}. Call a function f : R n R Σ-differentiable in a R n if there are c... c n R such that 6

7 lim f(x+a) f(a) (c u + +c nu n) u 0u R Σ u 0. Let Σ be a signature a R n. A Σ-neighborhood of a is a subset U Σ (a) R n such that there is a (regular) neighborhood U(a) ith U Σ (a) U(a) (a+r Σ ) here a + R Σ a + u; u R Σ }. We say that f : R n R has partial derivatives in U Σ (a) if for each b U Σ (a) it holds: if b i a i for some i then f(b) x i exists and if b i a i for some i then Σ if(b) Σ ix i exists. Let f(x...x n ) : R n R have the derivatives f u + (a) in a R n for every u U R n. The vector d U such that f + d (a) minf + u ; u U} d u is (if exists) called the direction of the steepest descent in a ith respect to U. Here f u + (a) lim f(a+tu) f(a) t 0 + t. It holds that if f is Σ-differentiable in a then f has partial derivatives in U Σ (a). The properties necessary for the backpropagation like adaptation algorithm to be ell ustified are the folloing ones. The proofs follo from [2] our form of the error function and the fact that a sum of Σ-differentiable function is again a Σ-differentiable function. Complete proofs can be found in [3]. Theorem 5 ([3]) E(... (l) i...θ(l)... λ(l)...) is continuous. Theorem 6 ([3]) For a given p... (l) i...θ(l) ± for (l) i 0 Σ(l) i + } for (l) i follos: Σ (l) i for λ (l) 0 Σ θ(l) ±. Then E(... (l) i... θ(l)...λ(l)... let Σ be a signature determined as 0 Σλ(l) ± for λ (l) 0 Σ λ(l) + }...) is Σ-differentiable in p.... λ(l) Theorem 7 ([2]) Let I... n} f(x... x n ) : R n R be Σ-differentiable in a R n for each Σ ith Σ i ± for i I Σ i + } for i I. Let d d...d n be a vector determined as follos: For i I d i f(a) x i. For i I let d i 0 for f(a) d i f(a) d i + f(a) d i f(a) d i + f(a) 0 and + f(a) 0 for f(a) for f(a) for f(a) for f(a) > 0 and + f(a) 0 0 and + f(a) > 0 > 0 and + f(a) > 0 and + f(a) < 0 and f(a) > + f(a) < 0 and f(a) < + f(a). Then d is the direction of the steepest descent in a ith respect to R n. By previous statements a deepest descent adaptation algorithm in our case is as meaningful as the classical backpropagation. It follos from the previous considerations and theorems that the only thing remaining to be able to use the steepest descent approach is to get the partial derivatives of E. Since E is a sum of the functions α E p it is enough to derive the formulas for derivatives of α E p : The derivative of E equals then the sum of the derivatives of α E p. In hat follos e omit for simplicity the indicies denoting the appropriate α cuts and patterns i.e. e rite E instead of α E p similarly for E L E R y (l) L y(l) R z(l) L z(l) etc. First e derive formulas for computing E i. Suppose (l) i 0. Then e have 7

8 E i i + E R i i L L z (l) z (l) i + R R z (l) z (l) i L for λ (l) 0 R for λ (l) < 0 R for λ (l) 0 L for λ (l) < 0 (3) z (l) λ (l) y(l) ) for LR} z (l) L i y(l ) Li y (l ) Ri (l) R for (l) i > 0 for (l) i < 0 z i y(l ) Ri y (l ) Li for (l) i > 0 for (l) i < 0. No for the output layer (l r) e have L y (l) L o L R 0 hereas for the inner layer (l < r) L m (l+) k ( y (l+) y (l+) k k L + y (l+) y (l+) k k ) L holds. Furthermore Similarly L (l+) k 0 else for (l+) k > 0 z(l+) L (l+) k 0 else for (l+) k < 0. R m (l+) k ( y (l+) y (l+) k k R + y (l+) y (l+) k k ) R R (l+) k 0 else for (l+) k < 0 z(l+) R (l+) k 0 else for (l+) k > 0. To sum up denote 8

9 δ L(l) L L λ (l) y(l) L L ) δl(l) R R λ (l) y(l) R R ). Then e have i δ L(l) L y (l ) i + δ L(l) R y (l ) i (4) L for (l) i > 0λ(l) 0 or (l) i < 0λ(l) R for (l) i > 0λ(l) R if L L if R 0 < 0 or (l) i < 0λ(l) > 0 (5) (6) here for the output layer (l r) δ L(l) L and for the inner layer (l < r) (y (l) L o L)λ (l) y(l) L L ) δl(l) R 0 m δ L(l) l+ L k (δ L(l+) k + δ L(l+) k )λ (l) y(l) L L ) here m δ L(l) l+ R k (δ L(l+) k + δ L(l+) k )λ (l) y(l) ) k k (l+) k 0 else (l+) k for (l+) k for (l+) k 0 else. λ (l+) k > 0 λ (l+) k < 0 Similar formulas can also be derived for E R. E R i For the output layer (l r) e have i δ R(l) L y (l ) i + δ R(l) R y (l ) i (7) δ R(l) L 0 δ R(l) R (y (l) R o R)λ (l) y(l) R R ) 9

10 and for the inner layer (l < r) m l+ δ R(l) L k (δ R(l+) k + δ R(l+) k )λ (l) y(l) L L ) m l+ δ R(l) R k (δ R(l+) k + δ R(l+) k )λ (l) y(l) ) If (l) i 0 then for + E L i ( E L i + E R i (7) here e set as in (5) for (l) i > 0 ((l) i By an analogous ay e derive formulas for E E R ) the same formulas can be used except for (4) i < 0 (l) i > 0 (l) i < 0). θ (l) E. E θ (l) θ (l) + E R. θ (l) θ (l) δ L(l) L δ L(l) R E R θ (l) δ R(l) L δ R(l) R For λ (l) 0 e have E + E R. For the output layer (l r) e have (y (l) L o L)z (l) y(l) L L ) For the inner layer (l < r) E R (y (l) R o R)z (l) y(l) L L ) k (δl(l+) k m l+ ml+ k (δl(l+) k + δ L(l+) + δ L(l+) k )z (l) k )z (l) y(l) L y(l) L L ) + L ) E R m l+ k (δr(l+) k ml+ k (δr(l+) k + δ R(l+) + δ R(l+) k )z (l) k )z (l) y(l) L y(l) L L ) + L ) 0

11 Similarly as in the case of eights if λ (l) 0 then for + E L formulas as above can be used ith the exception that for + E L for λ (l) > 0 hereas for E L E R E L and + E R e set as in (3) for λ (l) < 0. E R the same e set as in (3) + E R Note that the derived formulas can be a little bit simplified hich is omitted in our paper because of its limited scope. 4 Possibilities of applications The presented fuzzy neural netork is an adaptive model mapping tupples of fuzzy sets to tupples of fuzzy sets. It can be applied in every situation here an input output behaviour of a system hich is partially described by input output pairs of fuzzy sets is to be approximated. In the folloing e concentrate on the systems described by the so called linguistic IF THEN rules. Complex systems hich are hardly to describe by traditional techniques can often be effectively described by means of fuzzy mathematics. Especially if there is only a linguistic description of the concerned problem expressed in the form of IF THEN rules methods of fuzzy logic have been proved to be very poerful. The expert knoledge of a system behaviour can often be represented by a set of IF THEN rules of the form IF X is A THEN Y is B. Both the antecedent as ell as the succedent can be of a more complex form. The linguistic expressions A and B (so called evaluating expressions) can be modeled by appropriate fuzzy sets A B from F CI (R). The hole knoledge about such a rule base is then to be aggregated into a suitable model R. For an input represented by the fuzzy set A an output B A R is then to be obtained by some method. Noadays there are several methods developed in the frame of the so called approximated reasoning [9]. Usually R is a fuzzy relation and is some proection operation. For a detailed discussion and formal analysis see e.g. [5]. Our model offers an alternative method: Design a netork and adapt it to the training set T consisting of the associations A B. Then for a given input represented by A e get an output B from the netork. The great advantage of fuzzy logic systems is their interpretability ust the point here neural netorks fail. On the other hand the advantageous property of neural netorks is their learning capability and generalization capability. It is very often the case that the linguistic description does not describe the problem completely. Rather there are both the partial linguistic description and the measured data set describing the system behavior at disposal. In such a case e can transform the knoledge (linguistically expressed and crisp) to the training set T design a netork and adapt it to T. Our model is able to be adapted to both fuzzy and crisp data simultaneously. The net can be then used to perform the approximate reasoning task or due to the evidenced good generalization property it should be able to generate a ne rule base or to complete the old one. Concerning further the approximation reasoning task of considerable importance is the computational complexity of the procedure performing the operation see [0 6] for the analysis of some cases. In general the complexity of increases ith the number of rules in the rule base in the todays methods of approximate reasoning. On the other hand the computational complexity of in the case of our netork does not depend on the number of rules. It depends only on the architecture of the netork i.e. on the number of layers and neurons in the respective layers. We afford to omit the particular formulas for the computational complexity hich are easy to derive.

12 5 0 o Figure 2: Inputs and corresponding output generated by the adapted netork. As a further advantageous property of our model can be considered the fact that due to the cut by cut processing of fuzzy sets it makes possible to get only a certain horizontal part of the output fuzzy set e.g. the top part hich is of considerable information value. This is not possible by the above discussed traditional methods. 5 Example In this section e present an example demonstrating interpolation capabilities of our model. We trained a 2 layer netork ith to inputs three neurons in the first layer and one neuron in the second layer. The training set contained the folloing patterns: T } here r denotes a fuzzy number about r. After the netork as adapted inputs 5 and 0 have been presented to the netork. The output o of the netork is depicted in Fig. 2. (A linear transform has been used not to be restricted to the output interval (0).) 6 Conclusions We have presented an adaptive feedforard neural netork model hich maps tupples of fuzzy sets to tupples of fuzzy sets. Our netork is fuzzified by the extension principle in a ay proposed in [4]. Hoever eights thresholds and parameters of neuron transfer function are crisp. As shon this leads to computational simple model ith theoretically ell ustified and relatively simple adaptation formulas based on the steepest descent approach. The steepest descent adaptation eliminates the need for heuristic adaptation hich as proposed e.g. in [8]. We have proposed applications in the approximate reasoning tasks and discussed the advantages and disadvantages of our model. From a point of vie of further theoretical investigation a very interesting property establishing both neural netorks and fuzzy systems as universal tools for various kinds of engineering applications is the universal approximation property. The authors in [5] negatively ansered the question hether neural netorks ith both fuzzy signals and eights can be universal approximators. The question of approximation capabilities of the presented netorks ith fuzzy signals is still not definitely ansered and is orth of further research. Both theoretical and application oriented investigations should sho further possibilities in the systematic combination of fuzzy systems and neural netorks. The aim of this paper as to contribute to this endeavour. 2

13 References [] Arbib M (Ed.) (995) The Handbook of Brain Theory and Neural Netorks. Cambridge Massachusetts London MIT Press [2] Bělohlávek R (997) Backpropagation for interval patterns. Neural Netork World. Int. J. on Neural and Mass Parallel Computing and Information Systems. 7-3: [3] Bělohlávek R (998) Netorks Processing Indeterminacy. PhD dissertation Ostrava (available at request) [4] Buckley J J Hayashi Y (992) Fuzzy Neural Nets and Applications. Fuzzy Sets and Artificial Intelligence. : 4 [5] Buckley J J Hayashi Y (994) Can fuzzy neural nets approximate continuous fuzzy functions? Fuzzy Sets & Systems. 6: 43 5 [6] Dvořák A (997) Computational Properties of Fuzzy Logic Deduction. In: Reusch B. (Ed.): Computational Intelligence. Theory and Applications. Proceedings of the 5th Fuzzy Days Dortmund pp Berlin Springer [7] Gupta M M Rao D H (994) On the principles of fuzzy neural netorks. Fuzzy Sets & Systems. 6: 8 [8] Hayashi Y Buckley J J Czogala E (992) Direct Fuzzification of Neural Netork and Fuzzified Delta Rule. Proc. of the 2nd Int. Conf. on Fuzzy Logic & Neural Netorks pp Iizuka Japan [9] Klir G J Yuan B (995) Fuzzy Sets and Fuzzy Logic. Theory and Applications. Upper Saddle River NJ Prentice Hall [0] Kóczy L T (995) Algorithmic aspects of fuzzy control. Int. J. of Approximate Reasoning. 2: [] Kosko B (99) Neural Netorks and Fuzzy Systems. Upper Saddle River NJ Prentice Hall [2] Kruse R Gebhardt J Klaonn F (995) Fuzzy Systeme. Stuttgart B. G. Teubner [3] Kufudaki O Hořeš J (99) PAB: Parameters adapting back propagation. Neural Netork World. Int. J. on Neural and Mass Parallel Computing and Information Systems. : [4] Novák V (989) Fuzzy Sets and Their Applications. Bristol Adam Hilger [5] Novák V (996) Paradigm formal properties and limits of fuzzy logic. Int. J. of General Systems. 24: [6] Pedrycz W (993) Fuzzy neural netorks and neurocomputations. Fuzzy Sets & Systems. 56: 28 [7] Rumelhart D E Hinton G E Williams R J (986) Learning representations by backpropagating errors. Nature. 323: [8] Šíma J (992) Generalized back propagation for interval training patterns. Neural Netork World. Int. J. on Neural and Mass Parallel Computing and Information Systems. 2:

14 [9] Šíma J (995) Neural expert systems. Neural Netorks. 8: