COMP9414/ 9814/ 3411: Artificial Intelligence. 14. Course Review. COMP3411 c UNSW, 2014

Transcription

1 COMP9414/ 9814/ 3411: Artificial Intelligence 14. Curse Review

2 COMP9414/9814/ s1 Review 1 Assessment Assessable cmpnents f the curse: Assignment 1 10% Assignment 2 8% Assignment 3 12% Written Eam 70% Eam Template is available n the curse Web page Eam Questins will be similar in style t the Tutrial Questins

3 COMP9414/9814/ s1 Review 2 Tpics Cvered Prlg Prgramming Envirnment/Agent Types Search Strategies Infrmed Search (A ) Game Playing Lgical Inference Planning Learning and Decisin Trees Perceptrns Neural Netwrks Cnstraint Satisfactin Prblems Reasning under Uncertainty Rbtics Additinal Thursday Tpics: Reactive Agents General Game Playing Evlutinary Cmputatin Reinfrcement Learning

4 COMP9414/9814/ s1 Review 3 Agent Mdel sensrs envirnment percepts actins? agent actuatrs

5 COMP9414/9814/ s1 Review 4 Envirnment types We can classify envirnments as: - Fully Observable vs. Partially Observable - Deterministic vs. Stchastic - Single-Agent vs. Multi-Agent - Episdic vs. Sequential - Static vs. Dynamic - Discrete vs. Cntinuus - Knwn vs. Unknwn - Simulated vs. Situated r Embdied

6 COMP9414/9814/ s1 Review 5 Reactive Agents Agent Perceptin Actin Envirnment

7 COMP9414/9814/ s1 Review 6 Braitenberg Vehicles (Thu nly) HATE FEAR

8 COMP9414/9814/ s1 Review 7 Path Search Agent Wrld Mdel discrete states and transitins Planning BFS, DFS, UCS Greedy Search, A* Search Perceptin Actin Envirnment

9 COMP9414/9814/ s1 Review 8 Path Search Algrithms General Search algrithm: add initial state t queue repeat: take nde frm frnt f queue test if it is a gal state; if s, terminate epand it, i.e. generate successr ndes and add them t the queue Search strategies are distinguished by the rder in which new ndes are added t the queue f ndes awaiting epansin.

10 COMP9414/9814/ s1 Review 9 Search Strategies BFS and DFS treat all new ndes the same way: BFS DFS add all new ndes t the back f the queue add all new ndes t the frnt f the queue (Seemingly) Best First Search uses an evaluatin functin f() t rder the ndes in the queue; we have seen ne eample f this: UCS f(n) = cst g(n) f path frm rt t nde n Infrmed r Heuristic search strategies incrprate int f() an estimate f distance t gal Greedy Search A Search f(n) = estimate h(n) f cst frm nde n t gal f(n)=g(n)+h(n)

11 COMP9414/9814/ s1 Review 10 Cmpleity Results fr Uninfrmed Search Breadth- Unifrm- Depth- Depth- Iterative Criterin First Cst First Limited Deepening Time O(b (d+1) ) O(b C /ε ) O(b m ) O(b l ) O(b d ) Space O(b (d+1) ) O(b C /ε ) O(bm) O(bl) O(bd) Cmplete? Yes 1 Yes 2 N N Yes 1 Optimal? Yes 3 Yes N N Yes 3 b = branching factr, d = depth f the shallwest slutin, m = maimum depth f the search tree, l = depth limit. 1 = cmplete if b is finite. 2 = cmplete if b is finite and step csts εwith ε>0. 3 = ptimal if actins all have the same cst.

12 COMP9414/9814/ s1 Review 11 Game Search Agent Wrld Mdel Adversarial agent Planning minima search α β pruning Perceptin Actin Envirnment

13 COMP9414/9814/ s1 Review 12 Minima Search MAX (X) MIN (O) X X X X X X X X X MAX (X) X O X O X O... MIN (O) X O X X O X X O X TERMINAL Utility X O X X O X X O X O X O O X X O X X O X O O

14 COMP9414/9814/ s1 Review 13 α-β pruning MAX 3 3 MIN X X

15 COMP9414/9814/ s1 Review 14 Cnstraint Satisfactin Prblems backtracking search enhancements t backtracking search lcal search hill climbing simulated annealing

16 COMP9414/9814/ s1 Review 15 Lgical Agent Wrld Mdel knwledge base and lgical inference Planning reslutin situatin calculus Perceptin Actin Envirnment

17 COMP9414/9814/ s1 Review 16 Lgic Prpsitinal Lgic First Order Lgic Lgic Prgramming Planning

18 COMP9414/9814/ s1 Review 17 Statistical Learning Agent Agent Wrld Mdel Planning Perceptin Actin Statistical Learning Envirnment

19 COMP9414/9814/ s1 Review 18 Ockham s Razr The mst likely hypthesis is the simplest ne cnsistent with the data. inadequate gd cmprmise ver-fitting Since there can be nise in the measurements, in practice need t make a tradeff between simplicity f the hypthesis and hw well it fits the data.

20 COMP9414/9814/ s1 Review 19 Decisin Tree Patrns? Nne Sme Full F T Hungry? Yes N Type? F French Italian Thai Burger T F Fri/Sat? T N Yes F T

21 COMP9414/9814/ s1 Review 20 Chsing an Attribute Patrns? Type? Nne Sme Full French Italian Thai Burger Patrns is a mre infrmative attribute than Type, because it splits the eamples mre nearly int sets that are all psitive r all negative. This ntin f infrmativeness can be quantified using the mathematical cncept f entrpy. A parsimnius tree can be built by minimizing the entrpy at each step.

22 COMP9414/9814/ s1 Review 21 Rsenblatt Perceptrn 1 2 w 1 s Σ g g(s) w 2 1, 2 are inputs w 0 =-th w s=w w 2 2 th 1, w 2 are synaptic weights 1 = w w w 0 th is a threshld w 0 is a bias weight g is transfer functin

23 COMP9414/9814/ s1 Review 22 Perceptrn Learning Rule Adjust the weights as each input is presented. recall: s=w w w 0 if g(s)=0 but shuld be 1, if g(s)=1 but shuld be 0, w k w k + η k w k w k η k w 0 w 0 + η w 0 w 0 η s s s+η(1+ k k s s s η(1+ k k therwise, weights are unchanged. (η>0 is called the learning rate) Therem: This will eventually learn t classify the data crrectly, as lng as they are linearly separable.

24 COMP9414/9814/ s1 Review 23 Multi-Layer Neural Netwrks Output units a i W j,i Hidden units a j W k,j Input units a k

25 COMP9414/9814/ s1 Review 24 Gradient Descent We define an errr functin E t be (half) the sum ver all input patterns f the square f the difference between actual utput and desired utput E = 1 2 (z t) 2 If we think f E as height, it defines an errr landscape n the weight space. The aim is t find a set f weights fr which E is very lw. This is dne by mving in the steepest dwnhill directin. w w η E w Parameter η is called the learning rate.

26 L L COMP9414/9814/ s1 Review 25 Prbability and Uncertainty Start with the jint distributin: cavity Lcavity tthache catch catch Ltthache catch catch Can cmpute cnditinal prbabilities: P( cavity tthache) = P( cavity tthache) P(tthache) = = 0.4

27 COMP9414/9814/ s1 Review 26 Bayes Rule Prduct rule P(a b) = P(a b)p(b) = P(b a)p(a) Bayes rule P(a b)= P(b a)p(a) P(b) Useful fr assessing diagnstic prbability frm causal prbability: P(Cause Effect)= P(Effect Cause)P(Cause) P(Effect) e.g., let M be meningitis, S be stiff neck: P(m s)= P(s m)p(m) P(s) = = Nte: psterir prbability f meningitis still very small!

28 COMP9414/9814/ s1 Review 27 Evlutinary Cmputatin (Thu nly) use principles f natural selectin t evlve a cmputatinal mechanism which perfrms well at a specified task. start with randmly initialized ppulatin repeated cycles f: evaluatin selectin reprductin + mutatin any cmputatinal paradigm can be used, with apprpriately defined reprductin and mutatin peratrs

29 COMP9414/9814/ s1 Review 28 Reinfrcement Learning Agent Agent Wrld Mdel Planning Perceptin Actin Reinfrcement Learning Envirnment

30 COMP9414/9814/ s1 Review 29 Q-Learning (Thu nly) Fr each s S, let V (s) be the maimum discunted reward btainable frm s, and let Q(s,a) be the discunted reward available by first ding actin a and then acting ptimally. Then the ptimal plicy is where π (s)=argma a Q(s,a) Q(s,a)=r(s,a)+γV (δ(s,a)) then V (s)=maq(s,a), a s Q(s,a)=r(s,a)+γmaQ(δ(s, a), b) b which allws us t iteratively apprimate Q by ˆQ(s,a) r+ γma b ˆQ(δ(s, a), b)

31 COMP9414/9814/ s1 Review 30 Beynd COMP9414/ 3411 COMP3431 Rbtic Sftware Architecture COMP9417 Machine Learning and Data Mining COMP4418 Knwledge Representatin and Reasning COMP9444 Neural Netwrks (?) COMP9517 Machine Visin (?) COMP4431 Game Design Wrkshp (?) 4th Year Thesis tpics

32 COMP9414/9814/ s1 Review 31 Hierarchical Evlutinary Cmputatin I have designed a new evlutinary prgramming language called HERCL which cmbines elements frm Linear Genetic Prgramming with stackbased peratins frm FORTH. The idea is t prmte versatility and re-usability f genetic material acrss different task dmains (functin apprimatin, classificatin, string prcessing, cntrl prblems, etc.) 0[4<1>3< 1<c+0<-+2>^=1:g~:!. 2{^{ v.>g~!:2^ 2{1{.>g:1}.<2}<1>2;!!] 1[0<^<v-3<+2#%!+4> g:=:. 0jv1;] 2[3>c0>1j 0{3{0}3}v=:c4>0j;!] 3[1#c.>c1>vis ^}s;<^<y2j.=:{^w; ] This culd prvide a gd platfrm fr a Summer r 4th Year Thesis prject.

33 COMP9414/9814/ s1 Review 32 Learning a Strategy fr Multi-Player Chess Anther pssible prject is t use the TreeStrap algrithm t learn an evaluatin functin fr a multi-player chess game called Duchess:

34 COMP9414/9814/ s1 Review 33 COMP9414/ 3411 Artificial Intelligence QUESTIONS?

35 COMP9414/9814/ s1 Review 34 COMP9414/ 3411 Artificial Intelligence GOOD LUCK!