Artificial Intelligence Study Manual for the Comprehensive Oral Examination

Transcription

1 Artificial Intelligence Study Manual for the Comprehensive Oral Examination Hendrik Fischer Formerly of: Artificial Intelligence Center The University of Georgia Athens, Georgia Brian A. Smith Artificial Intelligence Center The University of Georgia Athens, Georgia Vineet Khosla Formerly of: Artificial Intelligence Center The University of Georgia Athens, Georgia

2 Contents 1 General Remarks 1 2 Symbolic Programming - CSCI/ARTI Basics PROLOG Algorithms in PROLOG Algorithms in LISP Factorial Fibonacci Recursion Standard Recursion Tail Recursion Other Examples Logic - CSCI(PHIL) 4550/6550, PHIL Definitions for Predicate Logic Entailment = Model Checking Decidability Unification Equivalence, Validity and Satisfiability Inference Procedure Soundness or Truth Preservation Completeness Types of Inference Procedures Propositional or Boolean Logic First-Order Logic Defeasible Logic Components Rule conflict Default Logic Sample derivations SD derivation PD derivation Artificial Intelligence - CSCI(PHIL) 4550/ Overview What is AI? Strong AI vs. Weak AI

3 4.1.3 Strong methods v. Weak methods Turing Test Chinese Room Experiment Mind-Body Problem Frame Problem Problem-Solving Search Overview Uninformed Search Informed (Heuristic) Search Adversarial Search (Game Playing) Constraint Satisfaction Problems Other Logic Logical Agents Wumpus World Model Checking Horn Clauses Knowledge-Base Forward- and Backward-Chaining Resolution Intelligent Agents Simple Reflex Agents Model-Based Agents Goal-Based Agents Utility-Based Agents Learning Agents Ideal Rational Agents Environments Planning Overview Search vs. Planning Planning as Search Partial-Order Planning STRIPS Expert Systems - CSCI Overview What Are Expert Systems? Components of an Expert System

4 5.1.3 Deep vs. Shallow Expert Systems Why and When Using Expert Systems? Symbolic Computation Rule-Based Systems Production Systems Conflict Resolution Forward vs. Backward Chaining Turing Machines - Computational Power and Expressiveness Time Complexity Dealing With Uncertainty MYCIN-Style Certainty Factors Fuzzy Sets (Possibility Theory) Conditional Probabilities (Bayesian) Dempster-Shafer Knowledge Representation Frames Semantic Networks / Associative Nets Ontologies Belief Revision / Truth Maintenance Systems Knowledge Acquisition Case-Based Reasoning Explanation Famous Problems Monty-Hall Problem Glasses of Water Intelligent Information Systems Blackboard Architecture Intelligent Software Agents Data Warehousing Active Database Data Mining Genetic Algorithms - CSCI(ENGR) 8940, CSCI(ARTI) Overview What is a Genetic Algorithm Components of a Genetic Algorithm Why Using Genetic Algorithms? No Free Lunch Theorem Intractable Problems Fundamentals

5 6.2.1 Theory of Evolution - Survival of the Fittest Schemata Building-Block Hypothesis Implicit Parallelism Importance of Diversity Premature Convergence Exponential Growth of Good Solutions Deception Baldwin Effect Genetic Operators Chromosome Representation Population Selection Crossover Mutation Advanced Genetic Algorithms Elitism Seeding Hill Climbing / Steepest Ascent Taboo Search Simulated Annealing Other Evolutionary Strategies Genetic Programming Evolutionary Programming Evolutionary Strategies Swarm Intelligence Neural Networks - CSCI(ENGR) 8940, CSCI(ARTI) Overview What is a (Artificial) Neural Network? Components of a Neural Network Why Using Neural Networks? Fundamentals Concept of a Perceptron Inductive Bias Learning From Example Noisy Data Overfitting Structure Layers

6 7.3.2 Net Input Activation Function Types of ANN (Supervised Learning) Feed-Forward NN Backpropagation NN Functional Link NN Product Unit NN Simple Recurrent NN Time Delay NN Hidden Units - What Are They Good For? Unsupervised Learning Self-Organizing Maps Machine Learning - CSCI(ARTI) Overview What is (Machine) Learning? Concept Learning Inductive Learning Hypothesis Concept Learning As Search General-To-Specific Ordering of the Hypothesis Space Find-S Algorithm Version Space Candidate Elimination Inductive Bias Futility of Bias-Free Learning Decision Trees Why prefer short hypotheses? Overfitting Entropy and Information Gain Bayesian Learning Bayes Rule Bayes Optimal Classifier Gibbs Classifier Naive Bayes Classifier Philosophy of Language PHIL(LING) 4300/ Fuzzy Logic - CSCI(ENGR) Overview What is Fuzzy Logic?

7 Components of Fuzzy Logic Why Using Fuzzy Logic? Fuzzy Systems Fuzzy Sets / Membership Functions Fuzzy Inference Fuzzy Controllers Mamdani Sugeno Knowledge Representation - PSYC Decision Support Systems Rapid Application Development Advanced Data Management (XML) Actual Questions 77

8 Abstract Most of this guide is written by Hendrik except for few sections written by Vineet. You are free to use this guide and make changes to it. The L A TEX source file can be obtained by ing either author. We are not going to charge anything but a few beers would be nice :) Best of Luck! 1 General Remarks 1. Be able to write short programs in PROLOG and LISP 2. Explain different programming concepts like tail recursion, recursion vs. iteration, working with lists (cdr/cad) 3. You won t have to formally prove any equations or formulas 4. Have examples ready for everything (e.g. a known toy problem to be solved by a PROLOG program) 5. Explain soundness and completeness and that kind of stuff about logic 6. What is Occam s Razor? 2 Symbolic Programming - CSCI/ARTI Basics PROLOG Ques 1. What are the four terms of Prolog? Atom = begin with lower case and includes all predicates. Structures = atom (arg1, arg2,..., argn). Variables = begin with uppercase or underscore. Numbers Ques 2. What isnegation by Failure? We cannot have true logical negation in Prolog because a Prolog is based on Horn Clause logic and in Horn clause we can have atmost one positive literal. Since we cannot state a negative fact, we instead try to get the positive fact to fail, hence having negation by failure. Ques 3. What is the difference between a red cut and a green cut? A green cut does not alter the output of the program but makes it more efficient. Red cut alters the possible output of the program. Ques 4. What are the advantages of using Prolog over other languages? 1

9 Inbuilt inference engine. Can allocate memory dynamically. Can modify itself. 2.2 Algorithms in PROLOG Subset of FOL - restricted to Horn Clauses Inference by resolution theorem proving Unification Depth-First-Search Backward-Chaining Approach Resolution is semi-decidable for FOL: KB = {P (f(x)) P (x)} and let s say we re trying to prove P (a). Resolution refutation assumes the negated goal - P (a) and from P (f(x)) P (x) we can derive P (f(a)) by substituting a for x. But then we can go on and resolve again to get P (f(f(a))) etc. Does this contradict the claim that resolution refutation is complete? NO - because completeness states that any sentence Q that can be derived by resolution from a set of premises P must be entailed by P. If it is not entailed by P, then we might no be able to prove that! Propositional calculus is decidable. Horn Clause: a clause (disjunction of literals) with at most one positive literal (in Prolog: q :- p 1, p 2,..., p n stands for the Horn Clause q p 1 p 2... p n Prolog syntax is based on Horn clauses, which have the form of p 1, p 2,..., p n q. In Prolog, this is mapped to an if-then -clause: q :- p 1, p 2,..., p n (q, if p 1 and p 2 and... ) This can be viewed as a procedure: (1) A goal literal matches (unifies) with q; (2) The tail of the clause p 1... p n is instantiated with the substitution of values for variables (unifiers) derived from this match; (3) The instantiated clauses serve as subgoals that invoke other procedures and so on. Theorem-proving method of Prolog is resolution refutation (assume the negated hypothesis and try to resolve the empty clause) - if you succeed, then it s safe to assert the hypothesis, otherwise not. Resolution simply works on contradictory literals (e.g. not p,q,p stands for if socrates is a man (p) then he is mortal (q) in CNF (p implies q becomes not p or q); 2

10 now if we assume that socrates is indeed a man, then by resolution, this implies q). If we have a goal q, then it might make sense to assert the negated goal for resolution refutation, which is essentially proof by reductio. This is a form of backward reasoning with sub-goaling: we assert the negated goal and try to work backwards, unifying and resolving clauses until we get to the empty clause, which allows us to claim that the Theory implies our original hypothesis (by reductio). Input = Query Q and a Logic Program P; Output = "yes" if Q follows from P, "no" otherwise; Initialize current goal set to {Q} While(the current goal set is not empty) do Choose a G from the current goal set (first) Look for a suitable rule in the knowledge-base; unify any variable if necessary to match rule; G :- B1, B2, B3,... If (no such rule exists) EXIT Else Replace G with B1, B2, B3... If (current goal set is empty) return NO. Else return YES. (plus all unifications of variables) Cuts (!) in Prolog are NOT sound! Prolog itself (even if not considering the failure by negation (\ +) and the cut is NOT complete. Prolog uses Linear Input Resolution (resolve axiom and goal, then axiom and newly resolved subgoal, etc. + never resolve two axioms). 2.3 Algorithms in LISP CDR yields rest of the list, CAR yields head of the list. Lisp programs are S-expressions containing S-expressions. So basically, a LISP program is a 3

11 linked list processing linked lists. 2.4 Factorial factorial(1,1). factorial(n,result) :- NewN is N-1, factorial(newn,newresult), Result is N * NewResult. tail recursion: factorial(n,result) :- fact_iter(n,result,1,1). fact_iter(n,result,n,result) :-!. fact_iter(n,result,i,temp) :- I < N, NewI is I+1, NewTemp is Temp * NewI, fact_iter(n,result,newi,newtemp). (defun factorial (n) (cond ((<= n 1) 1) (* n (factorial (- n 1))))) (defun fact_tail (n) (fact_iter 1 1 n)) (defun fact_iter (result counter max) (cond ((> counter max) result) (T (fact_iter (* result counter) (+1 counter) max)))) 2.5 Fibonacci fib(1,1). fib(2,1). fib(n,result) :- N > 2, N1 is N-1, N2 is N-2, fib(n1,f1), 4

12 fib(n2,f2), Result is F1 + F2. tail: fibo(n,result) :- fib_iter(n,result,1,1,1). fib_iter(n,result,n,_,result) :-!. fib_iter(n,result,i,f1,f2) :- I < N, NewI is I+1, NewF1 is F2, NewF2 is F1+F2, fib_iter(n,result,newi,newf1,newf2). (defun fib (n) (cond ((<= n 2) 1) (+ (fib (- n 1)) (fib (- n 2))))) (defun fib_tail (n) (fib_iter n)) (defun fib_iter (result counter f1 f2 max) (cond ((> counter max) result) (T (fib_iter (+ f1 f2) (+1 counter) f2 (+ f1 f2) max))) 2.6 Recursion Standard Recursion Program calls itself recursively from within the program. Uses a stack to store intermediate states that wait for a result from a lower level. The recursion goes down to a stopping condition, which returns a definite result that is passed on to the next higher level until we reach the top level again. The final result is computed along the way. 5

13 2.6.2 Tail Recursion In tail recursion, the recursive call is the last call in the program and there are no backtrack points. This allows us to build up the result while we re going down in the recursion until we hit the bottom level; along the way, we compute the final result, such that it is available once we reach the stopping condition. Smart compilers recognize tail recursion and stop the program once we hit the bottom - but some compilers bubble the result up to the top level before returning it. 2.7 Other Examples % FLATTEN (defun flatten (thelist) (if ((null thelist) nil) ((atom thelist) (list thelist)) (t (append (flatten (car thelist))) (flatten (cdr thelist))))) % REVERSE (defun myrev (thelist) (if ((null thelist) nil) (append (myrev (cdr thelist)) (list (car thelist)) % reverse function...reverses a list...non tail recursive my_reverse([ ],[ ]). my_reverse([head Tail],Result) :- my_reverse(tail,revtail), append(revtail,[head],result). % reverse function...reverses a list...tail recursive fast_reverse(orignal,result):- nonvar(orignal), fast_reverse_aux(orignal,[ ],Result). fast_reverse_aux([head Tail],Stack,Result) :- fast_reverse_aux(tail,[head Stack],Result). 6

14 fast_reverse_aux([ ],Result,Result). % DELETE ELEMENT (Prolog) del(x,[],[]) :-!. del(x,[x Tail],Tail1) :-!, del(x,tail,tail1). del(x,[y Tail],[Y Tail1]) :- del(x,tail,tail1). % FLATTEN (Prolog) flat([],[]). flat([head Tail],Result) :- is_list(head), flat(head,flathead), flat(tail,flattail), append(flathead,flattail,result). flat([head Tail],[Head FlatTail]) :- \+(is_list(head)), flat(tail,flattail). % SET STUFF % Union union([],set,set). union([head Tail],Set,Union) :- member(head,set), union(tail,set,union). union([head Tail],Set,[Head Union]) :- \+(member(head,set)), union(tail,set,union). % PERMUTATION permut([],[]). permut([l H],R):- permut(h,r1),add(l,r1,r). add(x,l,[x L]). add(x,[l H],[L R]):- add(x,h,r). % POWER SET (the set of all possible subsets) power_set([],[]). power_set([head Tail],Result) :- power_set(tail,tailresult), power_set_aux(head,tailresult,[],tempresult), append(tailresult,tempresult,result). 7

15 power_set_aux(head,[s T],Stack,Result) :- is_list(s), append([head],s,r), power_set_aux(head,t,[r Stack],Result). power_set_aux(head,[s T],Stack,Result) :- \+ is_list(s), append([head],[s],r), power_set_aux(head,t,[r Stack],Result). power_set_aux(x,[],stack,[x Stack]). 3 Logic - CSCI(PHIL) 4550/6550, PHIL Definitions for Predicate Logic An atomic formula is a sentence letter or an n-place predicate, e.g. A or Rabx. A literal is an atomic formula or the negation of an atomic formula. A clause is a disjunction of literals. A unit clause is a disjunction with one disjunct / the same as a literal. A negative clause is a clause with no positive literals. A definite clause is a clause with exactly one positive literal. A Horn clause is a clause with at most one positive literal. An empty clause is a disjunction with no disjuncts. A ground term is a term that contains no variables, e.g. a constant or f((f(f(c)))). A ground literal is a literal with no variables. A ground clause is a clause with no variables / a disjunctions of ground literals Entailment = A set of sentences A semantically entails a sentence B iff in every model in which all sentences of A are true, B is also true or A = B. That is, every model of A is also a model of B. A set of sentences A logically entails a sentence B means that B can be 8

16 proven by A. That is, B can be derived from A by applying the according derivation steps (depends on the logic system used). A set of sentences A truth-functionally entails a sentence B iff there is no truth-value assignment (TVA) on which every member of A is true and B is false. Example: Wumpus world; Lets say that according to the KB, we know that there s a breeze in [2,1]. Hence possible models are: there is a pit in [1,3] and [2,2], or [2,2] only or [1,3] only. Then α 1 = There is no pit in [1,2] is entailed by the KB, because in every model in which KB is true, α is also true. If we consider α 2 = There is no pit in [2,2], then we can say that α 2 is not entailed by the KB, because in some models in which the KB is true, α 2 is false. Hence the agent cannot conclude that there is no pit in [2,2], nor can it conclude that there indeed IS a pit in [2,2]! Entailment is only semi-decidable in FOL (i.e. we can show that sentences follow from premises if they do, but we cannot always show it if they do not). To check an entailment claim, we have to check if for every model for p that it is also a model for q. This might be infeasible and that s why we use inference rules (derivations) to proof these claims (e.g. Modus Ponens, Resolution, etc.) Model Checking m is called a model for a sentence α if α is true on m. In propositional logic, a truth value assignment for a given sentence is an interpretation (similarly, in FOL, an interpretation is a definition of all terms, predicates etc.). Now, such an interpretation, or TVA, is a model for α iff α is true on that TVA. Model Checking is a type of infernece algorithm which enumerate all possible models to check if α is true for all models in which the knowledgebase is true. A model is generally a commitment to the truth of its components. 9

17 3.1.3 Decidability Decidable A set S is decidable iff there is an algorithm that determines for every input if the input is in the set or not. Decidable = recursive set (roughly speaking) In terms of turing machine a set S is recursive iff there is a Turing machine that for any input, halts and outputs 1 if the input is in S and halts and outputs 0 if the input is not in S. It can always tell you if something is a theorem and also if something is not a theorem. A QUESTION is decidable, semi-decidable, or undecidable. A SET or a FUNCTION is recursive, recursively enumerable, or neither. If a SET S is recursive, then the QUESTION Is x a member of S? is decidable, etc. Non-Decidable A set S is non-decidable iff it is not decidable. Semi-Decidable A set S is semi-decidable iff there is an algorithm that determines for every input if the input is in the set, but can t determine if an input is not in the set. Semi-Decidable=recursive enumerable. (roughly speaking) In terms of Turing machine a set S is recursive enumerable iff there is a Turing machine that for any input halt and outputs 1 if the input is in S and runs forever if the input is not in S. It can always tell you if something is a theorem, but not that something is not a theorem.???: Is the set of semi-decidable problems a subset of the non-decidable problems? Unification Unification takes two atomic sentences p and q and returns a substitution that would make p and q look the same or else fails. If we have Knows(John, Jane) in our KB and the sentence Knows(John, x), then we can unify both by substituting x with Jane in the second sentence, making them look alike. This is used for the generalized modus ponens. If we have the rule Knows(John, x) Hates(John, x), the above unification can be used to deduce that John hates Jane. 10

18 This is the main concept on which PROLOG is based on: the binding of uninstantiated variables with an atom or a term. The assignment with another uninstantiated variable (or even itself) is forbidden in modern PRO- LOG. This protection takes the form of an occurs-check (Otherwise A = f(a): A could be unified with f(f(f(... ))), which is not desirable) Equivalence, Validity and Satisfiability Two sentences α and β are logically equivalent if they are true in the same set of models. In other words, if α = β and β = α, then they are equivalent. A sentence is valid if it is true in all models in which the premises (or knowledge base) is true. For ex- A sentence that is true in all models is known as a tautology. ample, P P is valid and a tautology. A sentence that is not valid on any model is known as a contradiction. An example of such a sentence is P P. A sentence is satisfiable if it is true in some model. Validity and satisfiability are connected in the following manner: α is valid iff α is unsatisfiable. α = β iff the sentence (α β) is unsatisfiable. The above is called proof by refutation or proof by contradiction. 3.2 Inference Procedure Patterns of inference that can be applied to derive chains of conclusions that lead to a desired goal Soundness or Truth Preservation A derivation system is sound iff if Γ P, then Γ = P, for all Γ, P. Soundness is a property of an inference procedure and an inference procedure is sound iff derivations only produce entailed sentences. That is, a system is sound if all derived results are entailed by the knowledge base used in the derivation. If the system (claims to) prove something true, it really is true! That is, if the conclusion is true in all cases where the premises are true. This is sometimes called truth-preserving. 11

19 A logical argument is sound iff the argument is valid and all of its premises are true. An unsound inference rule is abduction (a b, and b, then a might be true, or not). All men are mortal. Socrates is a man Hence Socrates is mortal. This argument is valid and it is sound, since its premises are true. Whereas All animals can fly. Pigs are animals. Therefore, pigs can fly. is valid, but it is not sound, since the first premise is false. An example of a sound, but incomplete derivation system would be one containing only &I and &E. All derivable sentences in such a system are logically entailed, but there are many logically entailed sentences which are underivable Completeness A derivation system is complete iff if Γ = P, then Γ P, for all Γ, P. An inference procedure is complete iff derivations can produce all entailed sentences. That is, if something really is true, the system is capable of proving it. An example of a complete but unsound derivation system would be SD with the addition of abduction as an inference procedure. Such a system can, indeed, derive all logically entailed sentences. But, it can also derive sentences which are not logically entailed.???: Can SD with the addition of abduction as an inference procedure derive all sentences? 12

20 3.2.3 Types of Inference Procedures Modus Ponens P Q and P, therefore Q. Modus Ponens is sound. Modus Ponens is incomplete. Modus Ponens is complete for Horn KB s. Modus Tollens P Q and Q, therefore P. Modus Tollens is sound. And-Elimination P & Q, therefore P. And-Elimination is sound. And-Elimination is incomplete. Resolution Resolution is sound. Resolution is complete. Resolution works as a proof by reductio, based on a set of sentences in conjunctive normal form (CNF). If we got two sentences A B and A C in CNF, then we can use resolution to get B C. This also works with substituting ground terms for variables before the actual resolution. Proof by resolution refutation is assuming the negation of a hypothesis and then showing that we can resolve to the empty clause in some way (which essentially depicts a proof by reductio, or contradiction). Resolution is refutation complete means if the set of sentences is unsatisfiable, then you will always be able to derive a contradiction with resolution. 3.3 Propositional or Boolean Logic Ontological commitment: there exist only facts in the world. Epistemological commitment: true, false and unknown. Pros of Propositional Logic: (1) declarative - pieces of syntax correspond to facts. (2) compositional - the meaning of A B is derived from the meaning of its constituents A and B. (3) context-independent. Cons of Propositional Logic: (1) lacks the expressive power of higher logical languages like FOL (predicate logic). 13

21 Propositional Logic consists of atomic sentences (A, B, C... ) and a set of logical connectives (not, and, or, conditional, biconditional). Each atom can be assigned a truth value (TVA). The truth value of the main connective determines the meaning of the sentence (i.e. its truth value). 3.4 First-Order Logic Ontological commitment: there exist facts, objects and relations between those objects Epistemological commitment: true, false and unknown. FOL consists of variables (x, y, z... = variable terms), constants (a, b, c... = constant terms), first-order predicates (P, Q, R... = define a relation among objects of the universe of discourse), functions (legof, fatherof, +, *,... = take an object as input and return an object) and logical connectives (same as propositional logic + and quantifiers). Universal Instantiation: every instantiation of a universally quantified sentence is entailed by it Existential Instantiation: we can substitute a new constant (that does not appear anywhere else in the KB) for the existentially quantified variable of a sentence. This is called a Skolem constant. The new sentence is entailed by the old one. If we instantiate all sentences in all possible ways, we have reduced the FOL problem to propositional logic and can use its tools for inferences. FOL is semi-decidable: it stops if a formula follows, but could run forever without ever stopping if it does not. First-order logic is sound: means if K yields/derives P, then K entails P (K P, then K = P). Basically it means that it derives only entailed sentences. First-order logic is complete: If K entails P, then K yields/derives P (K = P, then K P). Basically it means that it can derive any sentence that is entailed. 14

22 3.5 Defeasible Logic Defeasible logic is a subset of non-monotonic logic. Non-monotonic logic - meaning: in monotonic logic, the line of reasoning usually is piling up propositions; but in non-monotonic logic, the propositions are defeasible: the number of valid propositions can decrease if justification for one of them is retracted. This line of reasoning resembles the scientific method (hypothesis, gather evidence, modify hypothesis according to new findings). Nonmonotonic reasoning: In Nonmonotonic reasoning we can reject an earlier conclusion on the basis of new information, even if the old conclusion was justified by the evidence we had at that time. This reasoning system lets us draw likely conclusions with less than conclusive evidence. Human reasoning is not monotonic. In the normal course of human reasoning we often arrive at conclusions which we later retract when additional evidence becomes available. The additional evidence defeats our earlier reasoning and much of our reasoning is nonmonotonic and Defeasible in this way???: Barring the retraction of rules, is it true that the set of entailed sentences can only increase asinformation is added to the knowledge base? Components Undercutting defeaters : Undercutting defeaters are too weak to support an inference on their own and their sole purpose is to call into question an inference we might otherwise be inclined to make. It is important to note here that they don t suggest the opposite conclusion or for that matter any conclusion at all. They stop you from making a wrong conclusion. Generally these rules can be identified by the word might. For example, A damp match might not burn. Strict rules : These are the rules which can never be defeated or undercut. They don t have exceptions and represent necessary truth conditions. An example would be All crows are black or Everyone born in Atlanta was born in Georgia. Defeasible rules : These are the weaker rules which can be defeated or undercut. For example Birds fly or Penguins live in Antarctica. Defeasible rules can be defeated or rebutted by a strict rule or another defeasible rule which supports the opposite inference. Defeasible rules can be undercut by undercutting defeaters which point out 15

23 a situation in which the rules they defeat might not apply.???: Can defeasible rules undercut one another, or is there a definite hierarchy? Rule conflict Strict Rule vs Defeasible Rule : The strict rule is always superior to the defeasible rule, even if the condition of the strict rule is defeasibly derivable while the condition of the defeasible rule is strictly derivable. Defeasible Rule vs Defeasible Rule : When two defeasible rules reach a contradicting conclusion, we need a way to break that loop. We can either declare competing rules as incompatible or declare one rule superior to other. Undercutting defeaters : Undercutting defeaters are used to defeat defeasible rules. Undercutting defeaters are small pieces of knowledge, which stop us from making conclusions we normally would have made. 3.6 Default Logic Behaves like standard logic if complete information is available, but lets us also infer things if information is not complete. For example, in standard logic, if we know that something is a bird, we don t infer that it can fly, because there are birds that don t fly. In default logic, we would infer that it flies, until we get evidence that it might be otherwise. If our initial premises are: Bird(Condor), Bird(Penguin), Flies(Penguin), Flies(Airplane) and our default rule is W = { } Bird(X):F lies(x) F lies(x) then we can safely infer that a condor flies, because condor is a bird and we don t have evidence that condors can t fly. However, we cannot infer that penguins fly, because it is inconsistent with our knowledge. 16

24 3.7 Sample derivations SD derivation Given assumptions 1-3 below, derive H J. 1 (H & T) J Assp 2 (M D) & ( D M) Assp 3 T ( D & M) Assp 4 H Assp 5 J Assp 6 D Assp 7 D M 2&E 8 M 6,7 E 9 M D 2&E 10 D 8,9 E 11 D 6-10 E 12 T Assp 13 H & T 4,12&I 14 J 1,13 E 15 J 5Reit 16 T I 17 D & M 3,16 E 18 D 17&E 19 J 5-18 E 20 H J 4-19 I PD derivation Derive ( x)(bi Ax) Bi ( x)ax. 1 ( x)(bi Ax) Assp 2 Bi Assp 3 Bi Ac 1 E 4 Ac 2,3 E 5 ( x)ax 4 I 6 Bi ( x)ax 2-5 I 7 Bi ( x)ax Assp 8 Bi Assp 9 ( x)ax 7,8 E 10 Ac 9 E 11 Bi Ac 8-10 I 12 ( x)(bi Ax) 11 I 17

25 13 ( x)(bi Ax) Bi ( x)ax 1-6,7-12 I 4 Artificial Intelligence - CSCI(PHIL) 4550/ Overview Except for the search part, almost everything else covered in this class is covered in detail in a specialized class e.g. expert systems, genetic algorithms, neural networks etc What is AI? Artificial Intelligence is the field that strives to manufacture machines that exhibit intelligence. This leads to two questions - what is artificial, i.e. what can be built, and what is intelligence or intelligent behavior. There are numerous definitions of what artificial intelligence is. We end up with four possible goals: 1. Systems that think like humans: Cognitive modeling (focus on reasoning and human framework) 2. Systems that think rationally: Laws of thought (focus on reasoning and a general concept of intelligence) 3. Systems that act like humans: Turing test (focus on behavior and human framework) 4. Systems that act rationally: Rational agent (focus on behavior and a general concept of intelligence) What is rationality? An ideal concept of intelligence, doing the right thing simply speaking. Definition: The art of creating machines that perform functions that require intelligence when performed by humans (Kurzweil) (1) Involves cognitive modeling - we have to determine how humans think in a literal sense (explain the inner workings of the human mind, which requires experimental inspection or psychological testing) - Newell and Simon: GPS (General Problem Solver). 18

26 (2) Deals with right thinking and dives into the field of logic. Uses logic to represent the world and relationships between objects in it and come to conclusions about it. Problems: hard to encode informal knowledge into a formal logic system + theorem provers have limitations (if there s no solution to a given logical notation). (3) Turing defined intelligent behavior as the ability to achieve humanlevel performance in all cognitive tasks, sufficient to fool a human person (=Turing Test). Avoided physical contact to the machine, because physical appearance is not relevant to exhibit intelligence. The total Turing Test includes this by encompassing visual input and robotics as well. (4) The rational agent - achieving one s goals given one s beliefs. Instead of focusing on humans, this approach is more general, focusing on agents (which perceive and act). More general than strict logical approach (=thinking rationally) Strong AI vs. Weak AI Strong AI = sentience : machine-based artificial intelligence that can truly reason and becomes self-aware (sentient), either human-like (thinks and reasons like a human mind) or non-human-like (different form of sentience and reasoning) - coined by John Searle: an appropriately programmed computer IS a mind (see Chinese Room). Weak AI = problem-tailored : machine-based artificial intelligence that can reason and solve problems in a limited domain. Hence it acts as if it were intelligent in that domain, but would not be truly intelligent or sentient Strong methods v. Weak methods Strong methods : Makes use of deep knowledge regarding the environment and possible situations that may be faced. Strong methods are usually finetuned to the problem at hand and make use of a problem model. Weak methods : Makes use of general problem solving structures such as logic and automated reasoning. Weak methods are not usually fine-tuned to a specific problem but are applicable to a broad range of different problem classes Turing Test A human judge engages in a natural language conversation with two other parties, one a human and the other a machine; if the judge cannot reliably 19

27 tell which is which, then the machine is said to pass the test. It is assumed that both the human and the machine try to appear human. Objections: (1) A machine passing the test could simulate human behavior, but this might be considered much weaker than intelligence - the machine might just follow a set of cleverly designed rules. Turing rebutted this by saying that maybe the human mind is just following a set of cleverly designed rules also. (2) A machine could be intelligent without being capable of conducting conversations with humans (3) Many humans might fail this test, if they re illiterate or inexperienced (e.g. children).???: How applicable is the third objection? Chinese Room Experiment Tried to debunk the claims of Strong AI. Searle opposed the view that the human mind IS a computer and that consequently, any appropriately construed computer program could also be a mind. Outline: Man who doesn t speak Chinese sits in a room and is handed sheets of paper with Chinese symbols through a slit. He has access to a manual with a comprehensive set of rules on how to respond to this input. The output is also in Chinese and to an observer on the outside of the room, the room appears to give perfectly reasonable responses to the inputs. Searle argues, that even though he can answer correctly, he doesn t understand Chinese; he s just following syntactical rules and doesn t have access to the meaning of them. Searle believes that mind emerges from brain functions, but is more than a computer program - it requires intentionality, consciousness, subjectivity and mental causation. One objection to Searle is the systems response. Of course the man doesn t know Chinese and neither does the room, but the man-room system clearly knows Chinese. This response points out the serious flaw in Searle s argument: he appears to require a single, irreducible part of the system that is responsible for intentionality. He refuses to allow for the possibility that intentionality emerges from the interaction of several parts. 20

28 4.1.6 Mind-Body Problem The problem deals with the relationship between mental and physical events. Suppose I decide to stand up. My decision could be seen as a mental event. Now something happens in my brain (neurons firing, chemicals flowing, you name it) which could be seen as a physical, measurable event. How can it be, that something mental causes something physical. So it s hard to claim that both are completely different things. One take on this is to say that these mental events are, in fact, physical events. My decision IS neurons firing, but I m not aware of this - I feel like I made a decision independently from the physical. Of course this works the other way round to: everything physical could, in fact, be mental events. When I stand up after having made the decision (mental event), I m not physically standing up, but my actions cause mental events in my and the bystanders minds - physical reality is an illusion. There are two extremes of ontological commitment in relation to this problem. Material reductionism holds that all mental activity is reducible to interactions taking place in an objective, physical reality. Radical idealism holds instead that physical reality is instead entirely reducible to mental activity in some consciousness. Descartes is famous for his treatment of the issue and his is sometimes known as Cartesian duality. It holds that mind (or spirit) is non-physical while the body is physical. Just how the mind and body could interact under such an interpretation was a mystery even to Descartes, though he did suggest that the pineal gland in the brain might somehow be responsible Frame Problem It is the question of how to determine which things remain the same in a changing environment Problem-Solving Occurs whenever an organism or artificial intelligence is at some current state and does not know how to proceed in order to reach a desired goal state. This is considered to be a problem that can be solved by coming up with a series of actions that lead to the goal state (the solving ). 21

29 4.2 Search Overview In general, search is an algorithm that takes a problem as input and returns with a solution from the search space. The search space is the set of all possible solutions. We dealt a lot with so called state space search where the problem is to find a goal state or a path from some initial state to a goal state in the state space. A state space is a collection of states, arcs between them and a non-empty set of start states and goal states. It is helpful to think of the search as building up a search tree - from any given node (state): what are my options to go next (towards the goal). Complete search : Search is complete if it is guaranteed to find a solution if a solution exists. Optimal search : A search method is optimal if it finds the cheapest solution. Complexity of search: It is of two types: 1. Time : How long does it take to find a solution. 2. Space : How much memory is used to find a solution. Name Complete Optimal Time Space Depth First Yes No O(b) m O(b m) Breadth First Yes Yes O(b) d O(b d ) Iterative Deepening Yes Yes O(b) d O(b d) Greedy Search No No O(b) m O(b) m A-Star Yes Yes Uniform Cost d = depth m = max depth if search tree is finite only when cost is uniform if heuristic is admisssible, i.e it never overestimates Uninformed Search Uninformed search (blind search) has no information about the number of steps or the path costs from the current state to the goal. They can only distinguish a goal state from a non-goal state. There is no bias to go 22

30 towards the desired goal. 1. Breadth-First-Search: Starts with the root node and explores all neighboring nodes and repeats this for every one of those (expanding the depth of the search tree by one in each iteration). This is realized in a FIFO queue. Thus, it does an exhaustive search until it finds a goal. BFS is complete (it finds the goal if one exists and the branching factor is finite). BFS is optimal (if it finds the node, it will be the shallowest in the search tree). Space: O(b d ). Time: O(b d ). Open list: nodes yet to be explored, closed list: nodes that have been explored. 2. Depth-First-Search: Explores one path to the deepest level and then backtracks until it finds a goal state. This is realized in a LIFO queue, or stack. DFS is complete (if the search tree is finite). DFS is not optimal (it stops at the first goal state it finds, no matter if there is another goal state that is shallower than that). Space: O(bm) (much lower than BFS). Time: O(b m ) (Higher than BFS if there is a solution on a level smaller than the maximum depth of the tree). Danger of running out of memory or running indefinitely for infinite trees. 3. Depth-Limited / Iterative Deepening: To avoid that, the search depth for DFS can be either limited to a constant value, depth-limited search. DLS is not complete, as the goal state may be deeper than the search depth. Iterative deepening Repeatedly applies depthlimited search, each time with an increased search depth. This is repeated until finding a goal. Iterative deepening combines the advantages of DFS and BFS. ID is complete. ID is optimal (the shallowest goal state will be found first, since the level is increased by one every iteration). Space: O(b d) (better than DFS, d is depth of shallowest goal state, instead of m, maximum depth of the whole tree). Time: O(b d ). 4. Bi-Directional Search: Searches the tree both forward from the initial state and backward from the goal and stops when they meet somewhere in the middle. Di-directional search requires a predecessor function in addition to the successor function required by all uninformed search methods. It is complete and optimal. 23

31 Space: O(b d 2 ). Time: O(b d 2 ). 5. Graph Search: Graph search avoids making repeated steps and is therefor useful when the search tree is a true tree, but rather a cyclic graph. Graph search maintains a closed list of expanded nodes and an open list of unexpanded nodes on the fringe. Graph search can be applied to any of the tree search algorithms previously discussed Informed (Heuristic) Search In informed search, a heuristic is used as a guide that leads to better overall performance in getting to the goal state. Instead of exploring the search tree blindly, one node at a time, the nodes that we could go to are ordered according to some evaluation function h(n) that determines which node is probably the best to go to next. This node is then expanded and the process is repeated ( Best First Search ). A* is a form of BestFS. In order to direct the search towards the goal, the evaluation function must include some estimate of the cost to reach the closest goal state from a given state. This can be based on knowledge about the problem domain and the description of the current node, the search cost up to the current node. BestFS optimizes DFS by expanding the most promising node first. Efficient selection of the current best candidate is realized by a priority queue. 1. Greedy Search: Minimizes the estimated cost to reach the goal. The node that is closest to the goal according to h(n) is always expanded first. It optimizes the search locally, but not always finds the global optimum. It is not complete (can go down an infinite branch of the tree), it is not optimal. Space: O(b m ) for the worst case, same for time, but can be reduced by choosing a good heuristic function. 2. A* Search: Combines uniform cost search (expand node on path with lowest cost so far) and greedy search. Evaluation function is f(n) = g(n) + h(n) (or estimated cost of the cheapest solution through node n). g(n) = Cost to reach the node. h(n) = Cost to get from node to goal. It can be proven that A* is complete and optimal if h is admissible - that is, if it never overestimates the cost to reach the goal. This is optimistic, 24

32 since they think the cost of solving the problem is less than it actually is. Examples for h: Path-Finding in a map: Straight-Line-Distance. 8-Puzzle: Manhattan-Distance to Goal State. Everything works, it just has to be admissible (e.g. h(n) = 0 always works, but transforms A* back to uniform cost search). If a heuristic function h 1 always estimates the actual distance to the goal better than another heuristic function h 2, then h 1 dominates h 2. A* maintains an open list (priority queue) and a closed list (visited nodes). If a node is expanded that s already in the closed list, stored with a lower cost, the new node is ignored. If it was stored with a higher cost, it is deleted from the closed list and the new node is processed. h is monotonic, if h(n) h(n ) g(n ) g(n), so that the difference between the heuristics of any two connected nodes does not overestimate the actual distance between those nodes. If h is monotonic and there is only one goal state, then h must be admissable. Example of a non-monotonic heuristic: n and n are two connected nodes, where n is the parent of n. Suppose g(n) = 3 and h(n) = 4, then we know that the true cost of a solution path through n is at least 7. Now suppose that g(n ) = 4 and h(n ) = 2. Hence, f(n ) = 6. First off, the difference in the heuristics (2) overestimates the actual cost between those nodes (1). However, we know that any path through n is also a path through n and we already know that any path through n has a true cost of at least 7. Thus, the f-value of 6 for n is meaningless and we will consider its parent s f-value. h is consistent, if the h-value for a given node is less or equal than the actual cost from this node to the next node plus the h-value from this next node (triangular inequality). If h is admissible and monotonic, the first solution found by A* is guaranteed to be the optimal solution (open/close list bookkeeping is no longer needed). Finding Heuristics: (a) Relax the problem (e.g. 8-puzzle: Manhattan distance). (b) Calculate cost of subproblems (e.g. 8-puzzle: calculate cost of first 3 tiles). 3. Uniform Cost Search: Use only the g value. Is optimal. 25