Fuzzy Logic. An introduction. Universitat Politécnica de Catalunya. Departament de Teoria del Senyal i Comunicacions.

Universitat Politécnica de Catalunya Departament de Teoria del Senyal i Comunicacions Fuzzy Logic An introduction Prepared by Temko Andrey

2 Outline History and sphere of applications Basics. Fuzzy sets and basic operations General structure of fuzzy system Fuzzy inferencing Fuzziness Vs Probability Comparison with other AI techniques (ANN) Classifier ensemble. Fuzzy integral

3 Origins and History 964 - Invented in USA by Lotfi Zadeh (Lotfi Aliaskerzadeh), native of Azerbaijan, also with Russian/Turkish/Iranian ancestry. 973 - First commercial application (cement mixing), Denmark. Lauritz Peter Holmblad, Jens-Jorgen Ostergaard. 973 - Application to steam engine control, U.K. Ebrahim Mamdani. 980's - Enthusiastically accepted in Japan, later in Singapore and other parts of east Asia.

4 History 980's - A large popular following in the West, but often sternly criticized or ridiculed by the scientific community. Relatively few applications in control (although a similar idea is used to process "confidence factors" in expert systems). 990's - Neuro-fuzzy applications very popular in Asia. 990's - Fuzzy control applications become widespread in the West. 992 992: st IEEE International Conference on Fuzzy Systems appearance of software companies (INFORM INFORM, Aptronix,etc.) Fuzzy Logic Toolbox for MATLAB was released in 994

5 Applications

5 Applications Control Algorithms Medical Diagnosis Decision Making Economics Engineering Environmental Literature

5 Applications Control Algorithms Medical Diagnosis Decision Making Economics Engineering Environmental Literature Pattern Recognition Psychology Reliability Security Science Operations Research

6 The Issue Question Why was a mathematical technique that was developed in the West, where mathematical modeling is widely used, largely rejected by Western science while it was enthusiastically applied in Asian countries? Western science has an ethic of difficulty or obscurantism, whereas fuzzy logic is simple. Answer It is hard to publish simple papers in Western journals. Some mechanism is needed to control access to the scientific establishment, and a preference for difficult and arcane contributions meets this need

7 Introduction So far as the laws of mathematics refer to reality, they are not certain. And so far as they are certain, they do not refer to reality. Albert Einstein Most of the phenomena we encounter everyday are imprecise - the imprecision may be associated with their shapes, position, color, texture, semantics that describe what they are Fuzziness primarily describes uncertainty (partial truth) and imprecision The key idea of fuzziness comes from the multivalued logic: Everything is a matter of degree Imprecision raises in several faces, e.g. as a semantic ambiguity

8 Precision and Significance REAL WORLD

9 Terminology Task: Define a set of young people

9 Terminology Task: Define a set of young people Crisp set defines membership of its elements from U: {0,}

9 Terminology Task: Define a set of young people Crisp set defines membership of its elements from U: {0,} Young 25

9 Terminology Task: Define a set of young people Crisp set defines membership of its elements from U: {0,} Young If x is 22 the degree of belongingness of x to the set Young is. 25

9 Terminology Task: Define a set of young people Crisp set defines membership of its elements from U: {0,} Young 25 If x is 22 the degree of belongingness of x to the set Young is. If x is 26 the degree of belongingness to Young is 0

0 Fuzzy sets Fuzzy set defines membership of its elements from U: [0,] 0.66

0 Fuzzy sets Fuzzy set defines membership of its elements from U: [0,] µ(x) 0.66 Young 0 25 30 40

0 Fuzzy sets Fuzzy set defines membership of its elements from U: [0,] µ(x) 0.66 Young 0 25 30 40 If x is 22 the degree of belongingness of x to the set Young is.

0 Fuzzy sets Fuzzy set defines membership of its elements from U: [0,] µ(x) 0.66 Young 0 25 30 40 If x is 22 the degree of belongingness of x to the set Young is. If x is 30 the degree of belongingness to Young is 0.666

Notations Discrete fuzzy set notation: A discrete fuzzy set A is denoted as the following x x 2 x n Discrete representation A x

Notations Discrete fuzzy set notation: A discrete fuzzy set A is denoted as the following x x 2 x n Discrete representation Continuous fuzzy set representation: A continuous fuzzy set A is denoted as the following A x

2 Operations. Union µ A B =max(µ A,µ B ) Membership A Membership B Membership A B

3 Operations. Intersection µ A B =min(µ A,µ B ) Membership A Membership B Membership A B

4 Complement µ A = - µ A Membership A Membership A

5 Other Operations Equality Product A=B if µ A (x)= µ A (x) µ AB (x) = µ A (x) µ B (x) Containment A is a subset of B if and only if µ A (x)<= µ B (x) Concentration Conc(µ A (x)) = µ A (x)^2 Proper Subset A is a proper subset of B if A is a subset of B and A<>B Dilation Dilation(µ A (x))= µ A (x)^0.5 etc..

6 Violations law of Excluded Middle law of Contradiction A A = U A A = Two fundamental laws of Classical Set theory are violated in Fuzzy Set Theory

7 Membership Functions Membership functions can be represented: graphically analytically in a tabular or list form geometrically(as a points in the unit cube)

8 Membership Functions (Ex.) All are just precise mathematical functions triangular trapezoidal sigmoid Gaussian-based

9 Linguistic Variables and Terms Linguistic variable {AGE} Linguistic terms {Young, Middle-aged, Old}

20 Fuzzy Rules General format: If x is A then y is B Examples: If pressure is high, then volume is small. If the road is slippery, then driving is dangerous. If a tomato is red, then it is ripe. If the speed is high, then apply the brake a little. each rule covers a local part of the whole working space

2 Fuzzy Inference Single rule with single antecedent Rule: if x is A then y is B Fact: x is A Conclusion: y is B

2 Fuzzy Inference Single rule with single antecedent Rule: if x is A then y is B Fact: x is A Conclusion: y is B Graphic Representation:

2 Fuzzy Inference Single rule with single antecedent Rule: if x is A then y is B Fact: x is A Conclusion: y is B Graphic Representation: A A X

2 Fuzzy Inference Single rule with single antecedent Rule: if x is A then y is B Fact: x is A Conclusion: y is B Graphic Representation: A A A X x is A X

2 Fuzzy Inference Single rule with single antecedent Rule: if x is A then y is B Fact: x is A Conclusion: y is B Graphic Representation: A A A X x is A X B Y

2 Fuzzy Inference Single rule with single antecedent Rule: if x is A then y is B Fact: x is A Conclusion: y is B Graphic Representation: A A A X x is A X B B y is B Y Y

22 Structure of Fuzzy System Crisp output Defuzzification Subsystem Control Process Fuzzy Inference Engine Fuzzy rule base Crisp input Fuzzification Subsystem

23 Example. Driving Problem: Driving on road, approached by vehicles towards an intersection. How do you control your breaks depending on the distance to the point of intersection and speed of the vehicle. Approach: Fuzzify the antecedent variables Fire the rules Obtain the net consequent membership function Defuzzify to get crisp output

24 Example. Initialization Defining input or antecedent membership functions: Near Far Slow Fast 0 0.3 0.7 Distance to intersection (dti) 0 0.2 0.8 Speed Defining output or consequent membership functions: Gentle Medium High 0 0.25 0.5 0.75 Pressure

25 Example. Fuzzy Rules Control Rules:. If the distance to intersection (dti) is far and the speed slow apply gentle breaks 2. If dti is far and the speed fast apply medium breaks 3. If dti is near and speed slow apply medium breaks 4. If dti is near and speed fast apply high breaks

26 Example. Input Given that the distance to intersection is 0.4 normalized units and speed is 0.7 normalized units. How much should be the pressure on the breaks in normalized units? Input: DTI = 0.4 Speed = 0.7 Pressure -? CRISP!!!!

27 Example. Fire the Rules Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 Slow Fast G M H 0 0.2 0.8 0 0.25 0.5 0.75 2 DTI is far and Speed is fast Pressure is medium

27 Example. Fire the Rules Input: DTI = 0.4 Speed = 0.7 Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 Slow Fast G M H 0 0.2 0.8 0 0.25 0.5 0.75 2 DTI is far and Speed is fast Pressure is medium

27 Example. Fire the Rules Input: DTI = 0.4 Speed = 0.7 Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 DTI is far Speed is slow Pressure is gentle and Slow Fast G M H 0 0.2 0.8 0 0.25 0.5 0.75 2 DTI is far and Speed is fast Pressure is medium

27 Example. Fire the Rules Input: DTI = 0.4 Speed = 0.7 Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 DTI is far Speed is slow Pressure is gentle and min Slow Fast G M H 0 0.2 0.8 0 0.25 0.5 0.75 2 DTI is far and Speed is fast Pressure is medium

27 Example. Fire the Rules Input: DTI = 0.4 Speed = 0.7 Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 DTI is far Speed is slow Pressure is gentle and min Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 2 DTI is far and Speed is fast Pressure is medium

28 Example. Fire the Rules Speed = 0.7 Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 DTI is near Speed is slow Pressure is medium 3 and Slow Fast G M H 0 0.2 0.8 0 0.25 0.5 0.75

28 Example. Fire the Rules Input: DTI = 0.4 Speed = 0.7 Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 DTI is near Speed is slow Pressure is medium 3 and Slow Fast G M H 0 0.2 0.8 0 0.25 0.5 0.75

28 Example. Fire the Rules Input: DTI = 0.4 Speed = 0.7 Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75 DTI is near Speed is slow Pressure is medium 3 and Near Far Slow Fast G M H 0 0.3 0.7 0 0.2 0.8 0 0.25 0.5 0.75

29 Example. Defuzzification G M H G M H G M H G M H 0 0.25 0.5 0.75 0 0.25 0.5 0.75 0 0.25 0.5 0.75 0 0.25 0.5 0.75 G M H 0 0.25 0.5 0.75 0.68 CRISP!!!!

30 Defuzzification Defuzzification is a process of conversion of fuzzy output (possibility distribution of the output) to precise (crisp) value Among the major defuzzification techniques we can mention: a) Mean of Maximum (Middle of Maxima) method (MoM MoM) b) Centroid (Center of Area, Center of Gravity) method (CoA CoA) MoM calculates the average of all variable values having maximum membership degree y defuzzified output CoA calculates the weighted average of the fuzzy output y y = = Y i µ B' µ ( B' y ( ) y ydy Y µ i ) y i µ B' ( y i B' ( y ) dy i ) a b output y

3 Fuzziness Vs Probability Probability theory is one of the most traditional theories for representing uncertainty in mathematical models Nature of uncertainty in a problem is a point which should be clearly recognized by engineer - there is uncertainty that arises from chance, from imprecision, from a lack of knowledge, from vagueness, from randomness probability theory deals with the expectation of an event (future event, its outcome is not known yet), i.e. it is a theory of random events

32 Fuzziness Vs Probability Fuzziness deals with the impression of meaning of concepts expressed in natural language - it is not concerned with events at all Fuzzy theory handles nonrandom uncertainty Random Uncertain Certain Fuzzy, imprecise, vague

33 Fuzziness Vs Probability (Ex.) tomorrow slight rains are highly probable Uncertainty Fuzziness Uncertainty Probability Probability and fuzziness are related but different concepts. Fuzziness is a type of deterministic uncertainty. It describes the event class ambiguity. Fuzziness measures the degree to which an event occurs, not whether it occurs.

34 Fuzzy Vs ANN Fuzzy ANN Advantages No training set of data required Conceptually easy to understand Capability to use expert knowledge expert kowledge is necessary to construct the rules need validation Disadvantages Little or no a priori knowledge required little or no idea how it reaches its conclusion a training set of data and the corresponding conclusions are abolutely required. need validation

35 Fuzzy Vs ANN Techniques are complementary rather than competitive Adaptive Neuro-Fuzzy Inference system Neural networks extract fuzzy rules from numerical data tune membership functions of fuzzy systems Fuzzification Inference Defuzzification

36 Why to Use Fuzzy Logic? Fuzzy logic is conceptually easy to understand. Fuzzy logic is flexible. Fuzzy logic is tolerant of imprecise data. Fuzzy logic can model nonlinear functions of arbitrary complexity. Fuzzy logic can be built on top of the experience of experts. Fuzzy logic can be blended with conventional control techniques. Fuzzy logic is based on natural language.

37 Fuzzy Theory Fuzzy Theory Fuzzy Fuzzy Uncertainty& Mathematics Decision-Making Information Fuzzy Systems Fuzzy Logic & AI

38 Classifier Combinations Output of i-th classifier Support of L classifiers for class w j DP decision profile C number of classes L number of classsifiers where is called aggregation rule.

39 Classifier Combinations Non-fuzzy Non-trainable Majority vote Minimum, maximum, average, product etc... Fuzzy Fuzzy integral Decision template Trainable Naive Bayes Weighted majority vote MLP, etc...

40 Why Use Fuzzy Aggregation? Simple methods cannot not cover: Preferential dependence

40 Why Use Fuzzy Aggregation? Simple methods cannot not cover: Correlation Preferential dependence

40 Why Use Fuzzy Aggregation? Simple methods cannot not cover: Correlation Example Student evaluation Statistic Probability Algebra Preferential dependence

40 Why Use Fuzzy Aggregation? Simple methods cannot not cover: Correlation Redundancy Overestimation underestimation Example Student evaluation Statistic Probability Algebra Preferential dependence

40 Why Use Fuzzy Aggregation? Simple methods cannot not cover: Correlation Substitutiveness/complementarity Example Student evaluation Statistic Probability Algebra Preferential dependence

40 Why Use Fuzzy Aggregation? Simple methods cannot not cover: Correlation Substitutiveness/complementarity Example Example Student evaluation Statistic Probability Student evaluation Scientific subjects Algebra Literary subjects Preferential dependence

40 Why Use Fuzzy Aggregation? Simple methods cannot not cover: Correlation Substitutiveness/complementarity Example Student Example evaluation Statistic Example Student evaluation Probability Scientific subjects Algebra Literary subjects Preferential dependence

4 Fuzzy Measure Generalization of probability measures replacing the additive property by the monotonicity property with respect to set inclusion: FM: FM2: Probability, possibility, belief and plausibility measures are all special cases of fuzzy measure.

42 Example. Fuzzy Measure A student takes three subject maths (M), physics (P) and literature (L) Consider a fuzzy measure representing the level of importance associated with being good at various combinations of subjects: μ(m)= μ(p)=0.45, μ(l)=0.3 μ(m,l)= μ(p,l)=0.9>0.45+0.3 Taken independently being good in maths and physics is more important than being good at literature. It is rare for a student to be good at both maths and lit. or physics and lit. and indicates a versitile mind. μ(m,p)=0.5<0.45+0.45 Students who are good at maths are usually good at physics and vica-versa μ(m,p,l)=

43 Fuzzy Integral Choquet type Sugeno type where σ i = σ(x i ) ordered such that: based on MAX outputs of classifiers

43 Fuzzy Integral Choquet type Sugeno type where σ i = σ(x i ) ordered such that: based on SUM based on MAX outputs of classifiers

43 Fuzzy Integral Choquet type based on SUM Sugeno type where σ i = σ(x i ) ordered such that: based on MAX outputs of classifiers Fuzzy integral is COMPROMISE between COMPETENCE (expressed by fuzzy measure µ) and EVIDENCE (expressed by classifiers outputs σ)

44 Example. Choquet Integral For the school example suppose we have the following information regarding the student: σ(l)=0.4, σ(p)=0.7 and σ(m)=0.9 µ(m)= µ(p)=0.45, µ(l)=0.3, µ(m,l)= µ(p,l)=0.9 µ(m,p)=0.5 Choquet type

45 Example. Sugeno Integral For the school example suppose we have the following information regarding the student: σ(l)=0.4, σ(p)=0.7 and σ(m)=0.9 µ(m)= µ(p)=0.45, µ(l)=0.3, µ(m,l)= µ(p,l)=0.9 µ(m,p)=0.5 Sugeno type

46 The Problem Fuzzy measures require us to specify 2 n-2 pieces of information! In practice it s necessary to provide only fuzzy densities, i.e. µ(x i ) µ(m)= µ(p)=0.45, µ(l)=0.3, This has a unique non-zero solution λ > -

47 Example. Finding λ Recall the high school example: This generates the equation λ+=(+0.45λ)(+0.45 λ)(+0.3 λ) This has three solutions: µ(m)=µ(p)=0.45, µ(l)=0.3 λ={-7.33,-0.445, 0} Non-trivial solution λ=-0.445

48 Fuzzy Aggregation Vs MLP MLP is quite powerful in capturing complicated nonlinear relationship among varios inputs. Fuzzy-based methods provide interpretability of learned parameters. Using Shapley score we calculate the average impotance of each criteria in decision-making

49 Criteria Interaction Indices The averaged impotance is not overall impotance >> where This index is negative as soon as i and j are positively correlated or competitive. Similarly, it is positive when i and j are negatively correlated or complementary.

50 Conclusions No perfect classifier No perfect classifier fusion FL is not a remedy of all deseases What current work suggests is just to keep fuzzy logic and fuzzy combiners on the list of options and assign them high priority when choosing...