Outline. Definition of AI AI history Learning. Neural Network

Transcription

1 Artificial Intelligence An Introduction

2 Outline Definition of AI AI history Learning Q-learning Neural Network Genetic Algorithm

3 What is AI Thought Systems that Systems that process and think like humans think rationally reasoning Behavior Systems that act Systems that act like humans rationally Human performance Ideal concept of intelligence

4 Thinking humanly: Cognitive Modeling Cognitive Modeling is a interdisciplinary field bringing together computer models from AI and experimental technologies from psychology. 1960s cognitive revolution : requires scientific theories of internal activities of the brain What level of abstraction? How to validate? Top-down: predicting and testing behavior of human subject. Bottom-up: direct identifying from neurological data. Share with AI The available theories do not explain anything resembling human-level general intelligence.

5 Thinking rationally: Laws of Thought h (right thinking) Aristotle: what are correct arguments/thought g processes? Direct line through mathematics and philosophy p to AI Not all intelligent behavior is mediated by logical deliberation. What is the purpose of thinking? What thoughts should I have?

6 Acting humanly: The Turing Test Approach Turing (1950) computing machinery and intelligence (ch 26) Operational test for intelligent behavior. Predicted that by 2000, a machine might have a 30% chance of fooling a person for 5 minutes.

7 Acting rationally: Rational Agent Rational behavior: doing the right thing. The right thing: that which is expected to max goal achievement, given the available information. Watch example.

8 Rational Agents h An agent is an entity that perceives and acts.

9 AI prehistory Philosophy: logic, methods of reasoning mind as physical system, foundations of learning, language, g rationality Mathematics: formal representation and proof algorithms, computation, decidability, probability. Psychology: adaptation, phenomena of perception and motor control experimental techniques. Economics: formal theory of rational decisions. Linguistics: knowledge representation, grammar. Neuroscience: plastic physical substrate for mental activity Control theory: homeostatic systems, stability.

10 History of AI (I) Gestation ( ) 1955) McCulloch & Pitts (1943): boolean circuit model of brain. Minsky & Edmonds (1950): first neural network computer. Turing (1950): Computing Machinery and Intelligence.

11 History of AI (II) Birth (1956) Dartmouth workshop John McCarthy Minsky, Shannon, Rochester, More, Samuel, Solomonoff, Selfridge, Newell and Simon. AI from the start embraced the idea of duplicating i human faculties. AI is the only field to build machines that will function autonomously in complex, changing environment.

12 History of AI (III) Early enthusiasm, great expectations ( ) Samuel s checker program ram (1952). Newell & Simon s Logic Theorist (1956). We have invented a computer program capable of thinking non-numerically. Physical Symbol System (1976) A physical symbol system has the necessary and A physical symbol system has the necessary and sufficient means for general intelligence actions.

13 Early enthusiasm, great expectations (cont.) McCarthy (1958) Lisp Programs with Common Sense. Advice Taker first complete AI system. Knowledge Representation and Reasoning.

14 Early enthusiasm, great expectations (cond.) Minsky (1958) Microworlds block world Vision project - Huffman (1971) Constraint-propagation work - Waltz (1975) Learning theory Winston (1970) Natural language understanding di Winograd (1972) Planner - Fahlman (1974)

15 History of AI (IV) A dose of reality ( ) 1973) AI discovers computational complexity. Neural network research almost disappears. Simon (1957) It is not my aim to surprise or shock you but the simplest way I can summarize is to say that there are now in the world machines that think, that learn and that create. Moreover, their ability to do these things is going to increase rapidly until in a visible future the range of problems they can handle will be coextensive with the range to which the human mind has been applied.

16 A dose of reality (cont.) Most early programs contained little or no knowledge of their subject matter. The spirit is willing but the flesh is weak. The vodka is good but the meat is rotten. There has been no machine translation of general scientific text, and none is in immediate prospect. (1966) The intractability of many of the problems that AI was attempting to solve.

17 History of AI (V) AI becomes an industry (1980- present) Expert system st R1 first commercial expert system at DEC (1982). It was saving the company $40M/year. DuPont had 100 in use and 500 in development; saving $10M/year. Japanese announced the Fifth Generation project (1981).

18 History of AI (VI) The return of neural networks (1986- present) AI becomes a science (1987-present) Machine learning should not be isolated from information theory. Uncertainty reasoning stochastic modeling Search optimization theory Automated reasoning formal method and static analysis.

19 History of AI (VII) The emergence of Intelligent t Agents (1995-present) Realizing i that t the previously isolated subfields of AI might need to be reorganized when their results are to be tied together. AI has been drawn into much closer contact t with other fields, such as control theory and economics, that also deal with agents.

20 State of the Art Deep Blue defeated the reigning world chess champion Garry Kasparov in Proved a mathematical conjecture (Robbins conjecture) unsolved for decades. No hands across America (driving autonomously 98% of the time from Pittsburgh to San Diego). During the 1991 Gulf War, US forces deployed an AI logistics planning and scheduling program that involved up to 50,000 vehicles, cargo, and people. NASA's on-board autonomous planning program controlled the scheduling of operations for a spacecraft. Proverb solves crossword puzzles better than most humans.

21 State of the Art Play a decent game of table tennis, bridge,. Drive along a curving mountain road. Buy a week s worth of groceries at HEB, web. Discover and prove a new mathematical theorem. Translating spoken English to Swedish in real time. Perform a complex surgical operation..

22 Q-Learning

23 Q-Learning Unsupervised learning reinforcement learning Q-learning Modeling the Env Goal go outside of the building Acknowledgement:

24 State-Action Diagram

25 Reward to Action

26 Payoff Matrix R Action to go to state Agent now in state A B C D E F A B C D E F The minus sign in the table says that the row state has no action to go to column state.

27 Modeling the Agent The agent knows nothing about the ENV.

28 State-Action Diagram Again

29 Learning Matrix Q Learning rate The entry value in matrix Q (that is row represent state and column represent action) is equal to corresponding entry of matrix R added by a multiplication of a learning rate γ and maximum value of Q for all action in the next state. Note 0 γ 1. If γ is closer to 0, the agent will tend to consider only immediate reward. If γ is closer to 1, the agent will consider future reward with greater weight, willing to delay the reward.

30 Q-Learning Algorithm Input: State diagram with a goal state (represented by matrix R) Output: Minimum path from any initial state to the goal state (represented by matrix Q) Q Learning Algorithm Set γ, and environment reward matrix R Initialize matrix Q as zero matrix For each time-step: Select random initial state Do while not reach goal state Select one among all possible actions for the current state Using this possible action, consider to go to the next state Get maximum Q value of this next state based on all possible actions Compute the Q value Set the next state as the current state

31 The Execution Algorithm Input: Q matrix, initial state Output: t a sequence of current state t from initial iti state until goal state Set current state = initial state From current state, find action that produce maximum Q value Set current state = next state Go to 2 until current state = goal state

32 Training session 1 γ=0.8. Ag en t no w in st at e Initial state is room B. 1. Matrix Q at time step t=1. Action to go to state A B C D E F A B C D E F Age nt now in sta te Matrix R Action to go to state t A B C D E F A B C D E F

33 Training session 1 (cont.) 2. Random selection We select to go to F. Now we consider that suppose we are in state F. 3. Update Matrix Q Ag ent no w in sta te Action to go to state A B C D E F A B C D E F

34 Training session 2 Initial state is room D Random selection result Room B at t=1. Room F at t=2.

35 Training session 2 (cont.) Room B at t=1. Ag ent no w in sta te Action to go to state A B C D E F A B C D E F

36 Training session 2 (cont.) Room F at t=2. Ag ent no w in sta te Action to go to state A B C D E F A B C D E F Matrix Q isn t changed

37 After Multiple Training Sessions Action to go to state Age A B C D E F nt now A in B stat e C D E F

38 Normalization The Q Matrix, then can be normalized into a percentage by dividing all valid entries with the highest number (divided by 500 in this case) becomes Action to go to state Age A B C D E F nt now A in B stat t e C D E F

39 C-D-E-F Execution

40 Genetic Algorithms

41 Evolution Here s a very oversimplified description of how evolution works in biology Organisms (animals or plants) produce a number of offspring which are almost, but not entirely, like themselves Variation may be due to mutation (random changes) Variation may be due to sexual reproduction (offspring have some characteristics from each parent) Some of these offspring may survive to produce offspring of their own some won t The better adapted offspring are more likely to survive Over time, later generations become better and better adapted Genetic algorithms use this same process to evolve better programs

42 Components of a GA Population of individuals Individual is feasible solution to problem Each individual is characterized by a Fitness function Higher fitness is better solution Based on their fitness, parents are selected to reproduce offspring for a new generation Fitter individuals have more chance to reproduce New generation has same size as old generation; old generation dies Offspring has combination of properties of two parents If well designed, population will converge to optimal solution

43 The basic genetic algorithm Initialize with a large population of randomly generated attempted solutions to a problem Repeatedly do the following: Evaluate each of the attempted solutions. Select a subset of these solutions (the best ones). Use these solutions to reproduce a new population. Quit when you have a satisfactory solution (or you run out of time) reproduction parents deleted members population discard children evaluated children modification modified children evaluation

44 Population population Chromosomes could be: Bit strings ( ) Real numbers ( ) Permutations of element (E11 E3 E7... E1 E15) Lists of rules (R1 R2 R3... R22 R23) Program elements (genetic programming) g)... any data structure...

45 Reproduction reproduction children parents population Parents are selected at random with selection chances biased in relation to chromosome evaluations.

46 Chromosome Modification children modification modified children Modifications are stochastically triggered Operator types are: Mutation Crossover (recombination)

47 Mutation: Local Modification P1: ( ) C1: ( ) P1: ( ) C1: ( ) Causes movement in the search space (local or global) Restores lost information to the population

48 Crossover: Recombination Single point crossover P1 P2 Multi-point crossover P1 P2 ( ) ( ) C1 ( ) ( ) C2 ( ) ( ) C1 ( ) ( ) C2 Crossover greatly accelerates search early in evolution of a population It leads to effective combination of schemata (subsolutions on different chromosomes)

49 Evaluation evaluated children modified df d children evaluation The evaluator decodes d a chromosome and assigns it a fitness measure The evaluator is the only link between a The evaluator is the only link between a classical GA and the problem it is solving

50 Deletion discarded members population discard Generational GA: entire populations replaced with each iteration Steady-state GA: a few members replaced each generation

51 A really simple example Suppose your organisms are 32-bit computer words. You want a string in which all the bits are 1s. Here s how you can do it: Create 100 randomly generated computer words. Repeatedly do the following: Count the 1 bits in each word Exit if any of the words have all 32 bits set to 1 Keep the ten words that have the most 1s (discard the rest) From each word, generate 9 new words as follows: Pick a random bit in the word and toggle (change) it Note that this procedure does not guarantee that the next generation will have more 1 bits, but it s likely.

52 A more realistic example (1) Suppose you have a large number of (x, y) data points For example, (1.0, 4.1), (3.1, 9.5), (-5.2, 8.6),... You would like to fit a polynomial (of up to degree 5) through these data points That is, you want a formula y = ax 5 + bx 4 + cx 3 + dx 2 +ex + f that gives you a reasonably good fit to the actual data Here s the usual way to compute goodness of fit: Compute the sum of (actual y predicted y) 2 for all the data points The lowest sum represents the best fit There are some standard curve fitting techniques, but let s assume you don t know about them You can use a genetic algorithm to find a pretty good solution

53 A more realistic example (II) Your formula is y = ax 5 + bx 4 + cx 3 + dx 2 +ex + f Your genes are a, b, c, d, e, and f Your chromosome is the array [a, b, c, d, e, f] Your evaluation function for one array is: For every actual data point (x, y) Compute ý = ax 5 + bx 4 + cx 3 + dx 2 +ex + f Find the sum of (y ý) 2 over all x The sum is your measure of badness (larger numbers are worse) Example: For [0, 0, 0, 2, 3, 5] and the data points (1, 12) and (2, 22): ý= 0x 5 + 0x 4 + 0x 3 + 2x 2 +3x + 5 is = 10 when x is 1 ý= 0x 5 + 0x 4 + 0x 3 + 2x 2 +3x + 5 is = 19 when x is 2 (12 10) 2 + (22 19) 2 = = 13 If these are the only two data points, the badness of [0, 0, 0, 2, 3, 5] is 13

54 A more realistic example (III) Your algorithm might be as follows: Create 100 six-element arrays of random numbers Repeat 500 times (or any other number): For each of the 100 arrays, compute its badness (using all data points) Keep the ten best arrays (discard the other 90) From each array you keep, generate nine new arrays as follows: Pick a random element of the six Pick a random floating-point number between 0.0 and Multiply the random element of the array by the random floating-point number After all 500 trials, pick the best array as your final answer

55 An Exercise Find a pair of parameters that maximize the following equation: f(x, y) = 1/(1+x 2 +y 2 ) What are your genes? What s your fitness What are your genes? What s your fitness function?

56 An Exercise (II) Your formula is C = 1/(1+x 2 +y 2 ) Your genes are x and y Your chromosome is the array [x, y] The fitness function is simply C = 1/(1+x 2 +y 2 ) Create a pool of chromosomes X Y C0-1 2 C1-2 3 C C Fitness of initial population X Y C0-1 2 C1-2 3 C C Fitness Average

57 An Exercise (III) C1 is very small, discard it. X Y C0-1 2 C C Next generation with fitness evaluation. Select C3 and C2 as the first set of parents, and C3 and C0 as the second set of parents. Swap x and y in each chromosome. Calculate the new fitness values. X Y C0 (XC3YC2) C1 (XC2YC3)( ) C2 (XC3YC0) C3 (XC0YC3) -1-1 Fitness Average 0.39

58 (0,0) An Exercise (IV)

59 Neural Networks

60 Neural Networks Single Layer Perceptions u 1 u 2 w 1 w 2 γ O u 3 w 3 Input layer γ = w w 0 Output layer i=11 u i w i

61 Outputs Activation function Threshold function outputs 1 when the input is positive and 0 otherwise. +1 γ= 0, if (γ 0) γ= 1, if (γ>0) A neural network with a single hidden layer can represent any continuous function with arbitrary accuracy.

62 Multi-Layer Networks Adds a hidden layer, or multiple hidden layers, to increase the space of possible hypotheses the network can represent. w 3,1 u 1 u 3 w5,3 w 32 3,2 w 4,1 u 4 w 4,2 u 5 u 2 u 4 w 54 5,4 Input Hidden Output layer layer layer O

63 Activation function Outputs The sigmoid function is more flexible in that the outputs do not have to be 1 or 0, their outputs t are determined d by the sigmoid id function 1/(1 + exp(-γ)).

64 Issues The problem of choosing the number of hidden units is not well understood and optimal choices vary from problem to problem. Neural networks can be overfit, more is not necessarily better. In general, choose network and hidden layer size by trying several different ones and using the best.

65 Learning in Multilayer Networks Learning in multilayer l neural networks is similar to single layer networks. The problem is that it is less clear what value the hidden layers should output. The solution to this is the backpropagation on algorithm.

66 Back Propagation Simply a modification of perceptron learning algorithm adapted to incorporate multilayered networks with hidden units. Instead of a single output value, several outputs produce an output vector. When the output vector is compared with the training data, the algorithm back-propagates p the error from the output layer to the hidden layers.

67 Back Propagation Process Compute the change in values for the output units using the observed error. Starting with the output layer, repeat until earliest hidden layer is reached Propagate the change values back to the previous layer Update the weights between the layers