COMP9414/ 9814/ 3411: Artificial Intelligence 14. Curse Review
COMP9414/9814/3411 14s1 Review 1 Assessment Assessable cmpnents f the curse: Assignment 1 10% Assignment 2 8% Assignment 3 12% Written Eam 70% Eam Template is available n the curse Web page Eam Questins will be similar in style t the Tutrial Questins
COMP9414/9814/3411 14s1 Review 2 Tpics Cvered Prlg Prgramming Envirnment/Agent Types Search Strategies Infrmed Search (A ) Game Playing Lgical Inference Planning Learning and Decisin Trees Perceptrns Neural Netwrks Cnstraint Satisfactin Prblems Reasning under Uncertainty Rbtics Additinal Thursday Tpics: Reactive Agents General Game Playing Evlutinary Cmputatin Reinfrcement Learning
COMP9414/9814/3411 14s1 Review 3 Agent Mdel sensrs envirnment percepts actins? agent actuatrs
COMP9414/9814/3411 14s1 Review 4 Envirnment types We can classify envirnments as: - Fully Observable vs. Partially Observable - Deterministic vs. Stchastic - Single-Agent vs. Multi-Agent - Episdic vs. Sequential - Static vs. Dynamic - Discrete vs. Cntinuus - Knwn vs. Unknwn - Simulated vs. Situated r Embdied
COMP9414/9814/3411 14s1 Review 5 Reactive Agents Agent Perceptin Actin Envirnment
COMP9414/9814/3411 14s1 Review 6 Braitenberg Vehicles (Thu nly) HATE FEAR + + + +
COMP9414/9814/3411 14s1 Review 7 Path Search Agent Wrld Mdel discrete states and transitins Planning BFS, DFS, UCS Greedy Search, A* Search Perceptin Actin Envirnment
COMP9414/9814/3411 14s1 Review 8 Path Search Algrithms General Search algrithm: add initial state t queue repeat: take nde frm frnt f queue test if it is a gal state; if s, terminate epand it, i.e. generate successr ndes and add them t the queue Search strategies are distinguished by the rder in which new ndes are added t the queue f ndes awaiting epansin.
COMP9414/9814/3411 14s1 Review 9 Search Strategies BFS and DFS treat all new ndes the same way: BFS DFS add all new ndes t the back f the queue add all new ndes t the frnt f the queue (Seemingly) Best First Search uses an evaluatin functin f() t rder the ndes in the queue; we have seen ne eample f this: UCS f(n) = cst g(n) f path frm rt t nde n Infrmed r Heuristic search strategies incrprate int f() an estimate f distance t gal Greedy Search A Search f(n) = estimate h(n) f cst frm nde n t gal f(n)=g(n)+h(n)
COMP9414/9814/3411 14s1 Review 10 Cmpleity Results fr Uninfrmed Search Breadth- Unifrm- Depth- Depth- Iterative Criterin First Cst First Limited Deepening Time O(b (d+1) ) O(b C /ε ) O(b m ) O(b l ) O(b d ) Space O(b (d+1) ) O(b C /ε ) O(bm) O(bl) O(bd) Cmplete? Yes 1 Yes 2 N N Yes 1 Optimal? Yes 3 Yes N N Yes 3 b = branching factr, d = depth f the shallwest slutin, m = maimum depth f the search tree, l = depth limit. 1 = cmplete if b is finite. 2 = cmplete if b is finite and step csts εwith ε>0. 3 = ptimal if actins all have the same cst.
COMP9414/9814/3411 14s1 Review 11 Game Search Agent Wrld Mdel Adversarial agent Planning minima search α β pruning Perceptin Actin Envirnment
COMP9414/9814/3411 14s1 Review 12 Minima Search MAX (X) MIN (O) X X X X X X X X X MAX (X) X O X O X O... MIN (O) X O X X O X X O X............... TERMINAL Utility X O X X O X X O X O X O O X X O X X O X O O 1 0 +1...
COMP9414/9814/3411 14s1 Review 13 α-β pruning MAX 3 3 MIN 3 2 14 5 2 3 12 8 2 X X 14 5 2
COMP9414/9814/3411 14s1 Review 14 Cnstraint Satisfactin Prblems backtracking search enhancements t backtracking search lcal search hill climbing simulated annealing
COMP9414/9814/3411 14s1 Review 15 Lgical Agent Wrld Mdel knwledge base and lgical inference Planning reslutin situatin calculus Perceptin Actin Envirnment
COMP9414/9814/3411 14s1 Review 16 Lgic Prpsitinal Lgic First Order Lgic Lgic Prgramming Planning
COMP9414/9814/3411 14s1 Review 17 Statistical Learning Agent Agent Wrld Mdel Planning Perceptin Actin Statistical Learning Envirnment
COMP9414/9814/3411 14s1 Review 18 Ockham s Razr The mst likely hypthesis is the simplest ne cnsistent with the data. inadequate gd cmprmise ver-fitting Since there can be nise in the measurements, in practice need t make a tradeff between simplicity f the hypthesis and hw well it fits the data.
COMP9414/9814/3411 14s1 Review 19 Decisin Tree Patrns? Nne Sme Full F T Hungry? Yes N Type? F French Italian Thai Burger T F Fri/Sat? T N Yes F T
COMP9414/9814/3411 14s1 Review 20 Chsing an Attribute Patrns? Type? Nne Sme Full French Italian Thai Burger Patrns is a mre infrmative attribute than Type, because it splits the eamples mre nearly int sets that are all psitive r all negative. This ntin f infrmativeness can be quantified using the mathematical cncept f entrpy. A parsimnius tree can be built by minimizing the entrpy at each step.
COMP9414/9814/3411 14s1 Review 21 Rsenblatt Perceptrn 1 2 w 1 s Σ g g(s) w 2 1, 2 are inputs w 0 =-th w s=w 1 1 + w 2 2 th 1, w 2 are synaptic weights 1 = w 1 1 + w 2 2 + w 0 th is a threshld w 0 is a bias weight g is transfer functin
COMP9414/9814/3411 14s1 Review 22 Perceptrn Learning Rule Adjust the weights as each input is presented. recall: s=w 1 1 + w 2 2 + w 0 if g(s)=0 but shuld be 1, if g(s)=1 but shuld be 0, w k w k + η k w k w k η k w 0 w 0 + η w 0 w 0 η s s s+η(1+ k k s s s η(1+ k k therwise, weights are unchanged. (η>0 is called the learning rate) Therem: This will eventually learn t classify the data crrectly, as lng as they are linearly separable.
COMP9414/9814/3411 14s1 Review 23 Multi-Layer Neural Netwrks Output units a i W j,i Hidden units a j W k,j Input units a k
COMP9414/9814/3411 14s1 Review 24 Gradient Descent We define an errr functin E t be (half) the sum ver all input patterns f the square f the difference between actual utput and desired utput E = 1 2 (z t) 2 If we think f E as height, it defines an errr landscape n the weight space. The aim is t find a set f weights fr which E is very lw. This is dne by mving in the steepest dwnhill directin. w w η E w Parameter η is called the learning rate.
L L COMP9414/9814/3411 14s1 Review 25 Prbability and Uncertainty Start with the jint distributin: cavity Lcavity tthache catch catch.108.012.016.064 Ltthache catch.072.144 catch.008.576 Can cmpute cnditinal prbabilities: P( cavity tthache) = P( cavity tthache) P(tthache) 0.016 + 0.064 = 0.108+0.012+0.016+0.064 = 0.4
COMP9414/9814/3411 14s1 Review 26 Bayes Rule Prduct rule P(a b) = P(a b)p(b) = P(b a)p(a) Bayes rule P(a b)= P(b a)p(a) P(b) Useful fr assessing diagnstic prbability frm causal prbability: P(Cause Effect)= P(Effect Cause)P(Cause) P(Effect) e.g., let M be meningitis, S be stiff neck: P(m s)= P(s m)p(m) P(s) = 0.8 0.0001 0.1 = 0.0008 Nte: psterir prbability f meningitis still very small!
COMP9414/9814/3411 14s1 Review 27 Evlutinary Cmputatin (Thu nly) use principles f natural selectin t evlve a cmputatinal mechanism which perfrms well at a specified task. start with randmly initialized ppulatin repeated cycles f: evaluatin selectin reprductin + mutatin any cmputatinal paradigm can be used, with apprpriately defined reprductin and mutatin peratrs
COMP9414/9814/3411 14s1 Review 28 Reinfrcement Learning Agent Agent Wrld Mdel Planning Perceptin Actin Reinfrcement Learning Envirnment
COMP9414/9814/3411 14s1 Review 29 Q-Learning (Thu nly) Fr each s S, let V (s) be the maimum discunted reward btainable frm s, and let Q(s,a) be the discunted reward available by first ding actin a and then acting ptimally. Then the ptimal plicy is where π (s)=argma a Q(s,a) Q(s,a)=r(s,a)+γV (δ(s,a)) then V (s)=maq(s,a), a s Q(s,a)=r(s,a)+γmaQ(δ(s, a), b) b which allws us t iteratively apprimate Q by ˆQ(s,a) r+ γma b ˆQ(δ(s, a), b)
COMP9414/9814/3411 14s1 Review 30 Beynd COMP9414/ 3411 COMP3431 Rbtic Sftware Architecture COMP9417 Machine Learning and Data Mining COMP4418 Knwledge Representatin and Reasning COMP9444 Neural Netwrks (?) COMP9517 Machine Visin (?) COMP4431 Game Design Wrkshp (?) 4th Year Thesis tpics
COMP9414/9814/3411 14s1 Review 31 Hierarchical Evlutinary Cmputatin I have designed a new evlutinary prgramming language called HERCL which cmbines elements frm Linear Genetic Prgramming with stackbased peratins frm FORTH. The idea is t prmte versatility and re-usability f genetic material acrss different task dmains (functin apprimatin, classificatin, string prcessing, cntrl prblems, etc.) 0[4<1>3< 1<c+0<-+2>^=1:g~:!. 2{^{ v.>g~!:2^ 2{1{.>g:1}.<2}<1>2;!!] 1[0<^<v-3<+2#%!+4> g:=:. 0jv1;] 2[3>c0>1j 0{3{0}3}v=:c4>0j;!] 3[1#c.>c1>vis ^}s;<^<y2j.=:{^w; ] This culd prvide a gd platfrm fr a Summer r 4th Year Thesis prject.
COMP9414/9814/3411 14s1 Review 32 Learning a Strategy fr Multi-Player Chess Anther pssible prject is t use the TreeStrap algrithm t learn an evaluatin functin fr a multi-player chess game called Duchess:
COMP9414/9814/3411 14s1 Review 33 COMP9414/ 3411 Artificial Intelligence QUESTIONS?
COMP9414/9814/3411 14s1 Review 34 COMP9414/ 3411 Artificial Intelligence GOOD LUCK!