CS344: Introduction to Artificial Intelligence (associated lab: CS386)

Transcription

1 CS344: Introduction to Artificial Intelligence (associated lab: CS386) Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 1 to 6: Introduction; Fuzzy sets and logic; inverted pendulum 7 th, 8 th, 9 th 14 th, 15 th, 21 st Jan, 2013

2 Basic Facts Faculty instructor: Dr. Pushpak Bhattacharyya ( TAship: Kashyap, Bibek, Samiulla, Lahari, Jayaprakash, Nikhil, Kritika and Shuvam Course home page (will be up soon) Venue: LCC 02, opp KR bldg 1 hour lectures 3 times a week: Mon-10.30, Tue-11.30, Thu (slot 2)

3 Perspective

4 AI Perspective (post-web) Robotics NLP Expert Systems Search, Reasoning, Learning IR Planning Computer Vision

5 From Wikipedia Artificial intelligence (AI) is the intelligence of machines and the branch of computer science that aims to create it. Textbooks define the field as "the study and design of intelligent agents" [1] where an intelligent agent is a system that perceives its environment and takes actions that maximize its chances of success. [2] John McCarthy, who coined the term in 1956, [3] defines it as "the science and engineering of making intelligent machines." [4] The field was founded on the claim that a central property of humans, intelligence the sapience of Homo sapiens can be so precisely described that it can be simulated by a machine. [5] This raises philosophical issues about the nature of the mind and limits of scientific hubris, issues which have been addressed by myth, fiction and philosophy since antiquity. [6] Artificial intelligence has been the subject of optimism, [7] but has also suffered setbacks [8] and, today, has become an essential part of the technology industry, providing the heavy lifting for many of the most difficult problems in computer science. [9] AI research is highly technical and specialized, deeply divided into subfields that often fail to communicate with each other. [10] Subfields have grown up around particular institutions, the work of individual researchers, the solution of specific problems, longstanding differences of opinion about how AI should be done and the application of widely differing tools. The central problems of AI include such traits as reasoning, knowledge, planning, learning, communication, perception and the ability to move and manipulate objects. [11] General intelligence (or "strong AI") is still a long-term goal of (some) research. [12]

6 Topics to be covered (1/2) Search Logic General Graph Search, A*, Admissibility, Monotonicity Iterative Deepening, α-β pruning, Application in game playing Formal System, axioms, inference rules, completeness, soundness and consistency Propositional Calculus, Predicate Calculus, Fuzzy Logic, Description Logic, Web Ontology Language Knowledge Representation Semantic Net, Frame, Script, Conceptual Dependency Machine Learning Decision Trees, Neural Networks, Support Vector Machines, Self Organization or Unsupervised Learning

7 Topics to be covered (2/2) Evolutionary Computation Genetic Algorithm, Swarm Intelligence Probabilistic Methods Hidden Markov Model, Maximum Entropy Markov Model, Conditional Random Field IR and AI Modeling User Intention, Ranking of Documents, Query Expansion, Personalization, User Click Study Planning Deterministic Planning, Stochastic Methods Man and Machine Natural Language Processing, Computer Vision, Expert Systems Philosophical Issues Is AI possible, Cognition, AI and Rationality, Computability and AI, Creativity

8 Foundational Points Church Turing Hypothesis Anything that is computable is computable by a Turing Machine Conversely, the set of functions computed by a Turing Machine is the set of ALL and ONLY computable functions

9 Turing Machine Finite State Head (CPU) Infinite Tape (Memory)

10 Foundational Points (contd) Physical Symbol System Hypothesis (Newel and Simon) For Intelligence to emerge it is enough to manipulate symbols

11 Foundational Points (contd) Society of Mind (Marvin Minsky) Intelligence emerges from the interaction of very simple information processing units Whole is larger than the sum of parts!

12 Foundational Points (contd) Limits to computability Halting problem: It is impossible to construct a Universal Turing Machine that given any given pair <M, I> of Turing Machine M and input I, will decide if M halts on I What this has to do with intelligent computation? Think!

13 Foundational Points (contd) Limits to Automation Godel Theorem: A sufficiently powerful formal system cannot be BOTH complete and consistent Sufficiently powerful : at least as powerful as to be able to capture Peano s Arithmetic Sets limits to automation of reasoning

14 Foundational Points (contd) Limits in terms of time and Space NP-complete and NP-hard problems: Time for computation becomes extremely large as the length of input increases PSPACE complete: Space requirement becomes extremely large Sets limits in terms of resources

15 Two broad divisions of Theoretical CS Theory A Algorithms and Complexity Theory B Formal Systems and Logic

16 AI as the forcing function Time sharing system in OS Machine giving the illusion of attending simultaneously with several people Compilers Raising the level of the machine for better man machine interface Arose from Natural Language Processing (NLP) NLP in turn called the forcing function for AI

17 Allied Disciplines Philosophy Maths Economics Psychology Brain Science Knowledge Rep., Logic, Foundation of AI (is AI possible?) Search, Analysis of search algos, logic Expert Systems, Decision Theory, Principles of Rational Behavior Behavioristic insights into AI programs Learning, Neural Nets Physics Computer Sc. & Engg. Systems for AI Learning, Information Theory & AI, Entropy, Robotics

18 Goal of Teaching the course Concept building: firm grip on foundations, clear ideas Coverage: grasp of good amount of material, advances Inspiration: get the spirit of AI, motivation to take up further work

19 Resources Main Text: Artificial Intelligence: A Modern Approach by Russell & Norvik, Pearson, Other Main References: Principles of AI - Nilsson AI - Rich & Knight Knowledge Based Systems Mark Stefik Journals AI, AI Magazine, IEEE Expert, Area Specific Journals e.g, Computational Linguistics Conferences IJCAI, AAAI Positively attend lectures!

20 Grading Midsem Endsem Paper reading (possibly seminar) Quizzes

21 Modeling Human Reasoning Fuzzy Logic

22 Fuzzy Logic tries to capture the human ability of reasoning with imprecise information Works with imprecise statements such as: In a process control situation, If the temperature is moderate and the pressure is high, then turn the knob slightly right The rules have Linguistic Variables, typically adjectives qualified by adverbs (adverbs are hedges).

23 Alternatives to fuzzy logic model human reasoning (1/2) Non-numerical Non monotonic Logic Negation by failure ( innocent unless proven guilty ) Abduction (P Q AND Q gives P) Modal Logic New operators beyond AND, OR, IMPLIES, Quantification etc. Naïve Physics

24 Abduction Example If there is rain (P) Then there will be no picnic (Q) Abductive reasoning: Observation: There was no picnic(q) Conclude : There was rain(p); in absence of any other evidence

25 Alternatives to fuzzy logic model human reasoning (2/2) Numerical Fuzzy Logic Probability Theory Bayesian Decision Theory Possibility Theory Uncertainty Factor based on Dempster Shafer Evidence Theory (e.g. yellow_eyes jaundice; 0.3)

26 Linguistic Variables Fuzzy sets are named by Linguistic Variables (typically adjectives). Underlying the LV is a numerical quantity E.g. For tall (LV), height is numerical quantity. Profile of a LV is the plot shown in the figure shown alongside. µ tall (h) height h

27 Example Profiles µ rich (w) µ poor (w) wealth w wealth w

28 Example Profiles µ A (x) µ A (x) x Profile representing moderate (e.g. moderately rich) x Profile representing extreme

29 Concept of Hedge Hedge is an intensifier Example: LV = tall, LV 1 = very tall, LV 2 = somewhat tall 1 somewhat tall tall very operation: µ very tall (x) = µ 2 tall(x) µ tall (h) very tall somewhat operation: µ somewhat tall (x) = (µ tall (x)) 0 h

30 An Example Controlling an inverted pendulum: θ. d / dt = angular velocity Motor i=current

31 The goal: To keep the pendulum in vertical position (θ=0) in dynamic equilibrium. Whenever the pendulum departs from vertical, a torque is produced by sending a current i Controlling factors for appropriate current Angle θ, Angular velocity θ. Some intuitive rules If θ is +ve small and θ. is ve small then current is zero If θ is +ve small and θ. is +ve small then current is ve medium

32 Control Matrix θ. θ -ve med -ve small Zero +ve small +ve med -ve med -ve small +ve med +ve small Zero Region of interest Zero +ve small Zero -ve small +ve small Zero -ve small -ve med +ve med

33 Each cell is a rule of the form If θ is <> and θ. is <> then i is <> 4 Centre rules 1. if θ = = Zero and θ. = = Zero then i = Zero 2. if θ is +ve small and θ. = = Zero then i is ve small 3. if θ is ve small and θ. = = Zero then i is +ve small 4. if θ = = Zero and θ. is +ve small then i is ve small 5. if θ = = Zero and θ. is ve small then i is +ve small

34 Linguistic variables 1. Zero 2. +ve small 3. -ve small Profiles -ve small zero 1 +ve small -ε -ε ε ε ε +ε Quantity (θ, θ., i)

35 Inference procedure 1. Read actual numerical values of θ and θ. 2. Get the corresponding µ values µ Zero, µ (+ve small), µ (-ve small). This is called FUZZIFICATION 3. For different rules, get the fuzzy I-values from the R.H.S of the rules. 4. Collate by some method and get ONE current value. This is called DEFUZZIFICATION 5. Result is one numerical value of i.

36 Rules Involved if θ is Zero and dθ/dt is Zero then i is Zero if θ is Zero and dθ/dt is +ve small then i is ve small if θ is +ve small and dθ/dt is Zero then i is ve small if θ +ve small and dθ/dt is +ve small then i is -ve medium -ve small zero 1 +ve small -ε -ε ε ε ε +ε Quantity (θ, θ., i)

37 Fuzzification Suppose θ is 1 radian and dθ/dt is 1 rad/sec µ zero (θ =1)=0.8 (say) Μ +ve-small (θ =1)=0.4 (say) µ zero (dθ/dt =1)=0.3 (say) µ +ve-small (dθ/dt =1)=0.7 (say) -ve small zero 1 +ve small -ε -ε ε ε ε +ε 1rad 1 rad/sec Quantity (θ, θ., i)

38 Fuzzification Suppose θ is 1 radian and dθ/dt is 1 rad/sec µ zero (θ =1)=0.8 (say) µ +ve-small (θ =1)=0.4 (say) µ zero (dθ/dt =1)=0.3 (say) µ +ve-small (dθ/dt =1)=0.7 (say) if θ is Zero and dθ/dt is Zero then i is Zero min(0.8, 0.3)=0.3 hence µ zero (i)=0.3 if θ is Zero and dθ/dt is +ve small then i is ve small min(0.8, 0.7)=0.7 hence µ -ve-small (i)=0.7 if θ is +ve small and dθ/dt is Zero then i is ve small min(0.4, 0.3)=0.3 hence µ-ve-small(i)=0.3 if θ +ve small and dθ/dt is +ve small then i is -ve medium min(0.4, 0.7)=0.4 hence µ -ve-medium (i)=0.4

39 Finding i -ve medium -ve small zero 1 -ve small ε ε +ε 0.3 Possible candidates: i=0.5 and -0.5 from the zero profile and µ=0.3 i=-0.1 and -2.5 from the -ve-small profile and µ=0.3 i=-1.7 and -4.1 from the -ve-small profile and µ=0.3 -ε 2

40 Defuzzification: Finding i by the centroid method -ve medium ve small -2.5 Required i value Centroid of three trapezoids -ε +ε zero Possible candidates: i is the x-coord of the centroid of the areas given by the blue trapezium, the green trapeziums and the black trapezium

41 Fuzzy Sets

42 Theory of Fuzzy Sets Intimate connection between logic and set theory. Given any set S and an element e, there is a very natural predicate, µ s (e) called as the belongingness predicate. The predicate is such that, µ s (e) = 1, iff e S = 0, otherwise For example, S = {1, 2, 3, 4}, µ s (1) = 1 and µ s (5) = 0 A predicate P(x) also defines a set naturally. S = {x P(x) is true} For example, even(x) defines S = {x x is even}

43 Fuzzy Set Theory (contd.) Fuzzy set theory starts by questioning the fundamental assumptions of set theory viz., the belongingness predicate, µ, value is 0 or 1. Instead in Fuzzy theory it is assumed that, µ s (e) = [0, 1] Fuzzy set theory is a generalization of classical set theory aka called Crisp Set Theory. In real life, belongingness is a fuzzy concept. Example: Let, T = tallness µ T (height=6.0ft ) = 1.0 µ T (height=3.5ft) = 0.2 An individual with height 3.5ft is tall with a degree 0.2

44 Representation of Fuzzy sets Let U = {x 1,x 2,..,x n } U = n The various sets composed of elements from U are presented as points on and inside the n-dimensional hypercube. The crisp sets are the corners of the hypercube. µ A (x 1 )=0.3 (0,1) (1,1) µ A (x 2 )=0.4 U={x 1,x 2 } x 2 (x 1,x 2 ) x 2 A(0.3,0.4) (0,0) (1,0) Φ x 1 x 1 A fuzzy set A is represented by a point in the n-dimensional space as the point {µ A (x 1 ), µ A (x 2 ), µ A (x n )}

45 Degree of fuzziness The centre of the hypercube is the most fuzzy set. Fuzziness decreases as one nears the corners Measure of fuzziness Called the entropy of a fuzzy set Fuzzy set Farthest corner E( S) d( S, nearest) / d( S, farthest) Entropy Nearest corner

46 (0,1) (1,1) x 2 A (0.5,0.5) d(a, nearest) (0,0) x 1 (1,0) d(a, farthest)

47 Definition Distance between two fuzzy sets d( S, S ) n s ( x ) ( x i s2 i i 1 ) L 1 - norm Let C = fuzzy set represented by the centre point d(c,nearest) = = 1 = d(c,farthest) => E(C) = 1

48 Definition Cardinality of a fuzzy set n m( s) s ( x i ) (generalization of cardinality of i 1 classical sets) Union, Intersection, complementation, subset hood s s ( x) max( s ( x), s ( x)), x U s s ( x) min( s ( x), s ( x)), x U s c ( x) 1 ( x) s

49 Example of Operations on Fuzzy Set Let us define the following: Universe U={X 1,X 2,X 3 } Fuzzy sets A={0.2/X 1, 0.7/X 2, 0.6/X 3 } and B={0.7/X 1,0.3/X 2,0.5/X 3 } Then Cardinality of A and B are computed as follows: Cardinality of A= A = =1.5 Cardinality of B= B = =1.5 While distance between A and B d(a,b)= ) =1.0 What does the cardinality of a fuzzy set mean? In crisp sets it means the number of elements in the set.

50 Example of Operations on Fuzzy Set (cntd.) Universe U={X 1,X 2,X 3 } Fuzzy sets A={0.2/X 1,0.7/X 2,0.6/X 3 } and B={0.7/X 1,0.3/X 2,0.5/X 3 } A U B= {0.7/X 1, 0.7/X 2, 0.6/X 3 } A B= {0.2/X1, 0.3/X2, 0.5/X3} A c = {0.8/X 1, 0.3/X 2, 0.4/X 3 }

51 Laws of Set Theory The laws of Crisp set theory also holds for fuzzy set theory (verify them) These laws are listed below: Commutativity: A U B = B U A Associativity: A U ( B U C )=( A U B ) U C Distributivity: A U ( B C )=( A C ) U ( B C) De Morgan s Law: (A U B) C = A C B C A ( B U C)=( A U C) ( B U C) (A B) C = A C U B C

52 Distributivity Property Proof Let Universe U={x 1,x 2, x n } p i =µ AU(B C) (x i ) =max[µ A (x i ), µ (B C) (x i )] = max[µ A (x i ), min(µ B (x i ),µ C (x i ))] q i =µ (AUB) (AUC) (x i ) =min[max(µ A (x i ), µ B (x i )), max(µ A (x i ), µ C (x i ))]

53 Distributivity Property Proof Case I: 0<µ C <µ B <µ A <1 p i = max[µ A (x i ), min(µ B (x i ),µ C (x i ))] = max[µ A (x i ), µ C (x i )]=µ A (x i ) q i =min[max(µ A (x i ), µ B (x i )), max(µ A (x i ), µ C (x i ))] = min[µ A (x i ), µ A (x i )]=µ A (x i ) Case II: 0<µ C <µ A <µ B <1 p i = max[µ A (x i ), min(µ B (x i ),µ C (x i ))] = max[µ A (x i ), µ C (x i )]=µ A (x i ) q i =min[max(µ A (x i ), µ B (x i )), max(µ A (x i ), µ C (x i ))] = min[µ B (x i ), µ A (x i )]=µ A (x i ) Prove it for rest of the 4 cases.

54 Note on definition by extension and intension S 1 = {x i x i mod 2 = 0 } Intension S 2 = {0,2,4,6,8,10,..} extension

55 How to define subset hood?

56 Meaning of fuzzy subset Suppose, following classical set theory we say B A if ( x) ( x) x B A Consider the n-hyperspace representation of A and B (0,1) (1,1) x 2. B 1 A Region where ( x) ( x) B A.B 2.B 3 (0,0) x 1 (1,0)

57 This effectively means B P(A) CRISPLY P(A) = Power set of A Eg: Suppose A = {0,1,0,1,0,1,.,0,1} 10 4 elements B = {0,0,0,1,0,1,.,0,1} 10 4 elements Isn t B A with a degree? (only differs in the 2 nd element)

58 Subset operator is the odd man out AUB, A B, A c are all Set Constructors while A B is a Boolean Expression or predicate. According to classical logic In Crisp Set theory A B is defined as x x A x B So, in fuzzy set theory A B can be defined as x µ A (x) µ B (x)

59 Zadeh s definition of subsethood goes against the grain of fuzziness theory Another way of defining A B is as follows: x µ A (x) µ B (x) But, these two definitions imply that µ P(B) (A)=1 where P(B) is the power set of B Thus, these two definitions violate the fuzzy principle that every belongingness except Universe is fuzzy

60 Fuzzy definition of subset Measured in terms of fit violation, i.e. violating the condition B ( x) A( x) Degree of subset hood S(A,B)= 1- degree of superset = 1 x max(0, B ( x) m( B) A ( x)) m(b) = cardinality of B = x ( x) B

61 We can show that E( A) S( A A Exercise 1: c, A Show the relationship between entropy and subset hood Exercise 2: Prove that S( B, A) m( A B) / m( B) A c ) Subset hood of B in A

62 Fuzzy sets to fuzzy logic Forms the foundation of fuzzy rule based system or fuzzy expert system Expert System Rules are of the form If C C then C n A i Where C i s are conditions Eg: C 1 =Colour of the eye yellow C 2 = has fever C 3 =high bilurubin A = hepatitis

63 In fuzzy logic we have fuzzy predicates Classical logic P(x 1,x 2,x 3..x n ) = 0/1 Fuzzy Logic P(x 1,x 2,x 3..x n ) = [0,1] Fuzzy OR P( x) Q( y) max( P( x), Q( y)) Fuzzy AND P( x) Q( y) min( P( x), Q( y)) Fuzzy NOT ~ P( x) 1 P( x)

64 Fuzzy Implication Many theories have been advanced and many expressions exist The most used is Lukasiewitz formula t(p) = truth value of a proposition/predicate. In fuzzy logic t(p) = [0,1] t( P Q ) = min[1,1 -t(p)+t(q)] Lukasiewitz definition of implication

65 Fuzzy Inferencing Two methods of inferencing in classical logic Modus Ponens Given p and p q, infer q Modus Tolens Given ~q and p q, infer ~p How is fuzzy inferencing done?

66 A look at reasoning Deduction: p, p q - q Induction: p 1, p 2, p 3, - for_all p Abduction: q, p q - p Default reasoning: Non-monotonic reasoning: Negation by failure If something cannot be proven, its negation is asserted to be true E.g., in Prolog

67 Fuzzy Modus Ponens in terms of truth values Given t(p)=1 and t(p q)=1, infer t(q)=1 In fuzzy logic, given t(p)>=a, 0<=a<=1 and t(p >q)=c, 0<=c<=1 What is t(q) How much of truth is transferred over the channel p q

68 Lukasiewitz formula for Fuzzy Implication t(p) = truth value of a proposition/predicate. In fuzzy logic t(p) = [0,1] t( P Q ) = min[1,1 -t(p)+t(q)] Lukasiewitz definition of implication

69 Use Lukasiewitz definition t(p q) = min[1,1 -t(p)+t(q)] We have t(p->q)=c, i.e., min[1,1 -t(p)+t(q)]=c Case 1: c=1 gives 1 -t(p)+t(q)>=1, i.e., t(q)>=a Otherwise, 1 -t(p)+t(q)=c, i.e., t(q)>=c+a-1 Combining, t(q)=max(0,a+c-1) This is the amount of truth transferred over the channel p q

70 Two equations consistent Sub( B, A) 1 Sup( B, A) x U max(0, ( x ) ( x )) i 1 where U { x1, x2,..., xn} ( x ) x U i B i A i B i t( ( x ) ( x )) min(1,1 t( ( x )) t( ( x ))) B i A i B i A i These two equations are consistent with each other

71 Proof Let us consider two crisp sets A and B 1 U 2 U B A A B 3 U A B 4 U A B

72 Proof (contd ) Case I: ( x) 1 only when ( x) 1 So, ( x ) ( x ) 0 So, A i B i B i A i Sub( B, A) ( x ) x U i B i x U i max(0, ( x ) ( x )) x U i B i A i ( x ) B i

73 Proof (contd ) Since ( x ) ( x ) 0 B i A i L t( ( x ) ( x )) min(1,1 ( t( ( x )) t( ( x )))) B i A i B i A i min(1,1 ( ve)) 1 Thus, in case I these two equations are consistent with each other (prove for other cases)

74 Proof of the fact that S(B,A) is consistent with crisp set theory Case II (of the figure): ( x) 1 if, ( x) 1 So, ( x) ( x) 0 So, B i A i B i A i Sub( B, A) 1 x U i max(0, ( x ) ( x )) x U i B i A i ( x ) B i

75 Proof (contd ) ( x ) ( x ) 0 B i A i A( xi ) Sub( B, A) 1 1, ( x ) 0 Sub( B, A) 1 Thus, in case II also, these two equations are consistent with each other. B i

76 Proof of case III Case III: In This case, ( x) ( x) 0 for x ( B A) B i A i i and ( x ) ( x ) 0 otherwise B i A i So, Sub( B, A) 1 x U i max(0, ( x ) ( x )) x U i B i A i ( x ) B i

77 Proof (contd ) ( x ) ( x ) 0 only for x ( B A) B i A i i S( B, A) 1 B A A B B B Hence case III is also consistent across classical set theory and fuzzy set theory

78 Proof of case IV Case IV: In This case, ( x ) ( x) 1 for x A B i A i i, ( x) ( x) 1 for x B B i A i i and ( x) ( x) 0 otherwise B i A i So, Sub( B, A) 1 x U i max(0, ( x ) ( x )) x U i B i A i ( x ) B i

79 Proof (contd ) and ( x ) ( x ) 1 only for x B B i A i i <=0 otherwise, B S( B, A) 1 0 B Thus, in case IV also, these two equations are consistent with each other. Hence we can say that these two equations are consistent with each other in general.