Administrativia Assignment 1 due thursday 9/23/2004 BEFORE midnight Midterm eam 10/07/2003 in class CS 460, Sessins 8-9 1
Last time: search strategies Uninfrmed: Use nly infrmatin available in the prblem frmulatin Breadth-first Unifrm-cst Depth-first Depth-limited Iterative deepening Infrmed: Use heuristics t guide the search Best first: Greedy search queue first ndes that maimize heuristic desirability based n estimated path cst frm current nde t gal; A* search queue first ndes that maimize sum f path cst s far and estimated path cst t gal. Iterative imprvement keep n memry f path; wrk n a single current state and iteratively imprve its value. Hill climbing select as new current state the successr state which maimizes value. Simulated annealing refinement n hill climbing by which bad mves are permitted, but with decreasing size and frequency. Will find glbal etremum. CS 460, Sessins 8-9 2
Eercise: Search Algrithms The fllwing figure shws a prtin f a partially epanded search tree. Each arc between ndes is labeled with the cst f the crrespnding peratr, and the leaves are labeled with the value f the heuristic functin, h. Which nde (use the nde s letter) will be epanded net by each f the fllwing search algrithms? (a) Depth-first search (b) Breadth-first search (c) Unifrm-cst search (d) Greedy search (e) A* search 3 5 19 B h=14 C D 6 h=18 4 5 5 h=15 E F G H A h=10 h=12 h=8 h=10 h=20 CS 460, Sessins 8-9 3
Depth-first search Nde queue: initializatin # state depth path cst parent # 1 A 0 0 -- CS 460, Sessins 8-9 4
Depth-first search Nde queue: add successrs t queue frnt; empty queue frm tp # state depth path cst parent # 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 1 A 0 0 -- CS 460, Sessins 8-9 5
Depth-first search Nde queue: add successrs t queue frnt; empty queue frm tp # state depth path cst parent # 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 1 A 0 0 -- CS 460, Sessins 8-9 6
Depth-first search Nde queue: add successrs t queue frnt; empty queue frm tp # state depth path cst parent # 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 1 A 0 0 -- CS 460, Sessins 8-9 7
Eercise: Search Algrithms The fllwing figure shws a prtin f a partially epanded search tree. Each arc between ndes is labeled with the cst f the crrespnding peratr, and the leaves are labeled with the value f the heuristic functin, h. Which nde (use the nde s letter) will be epanded net by each f the fllwing search algrithms? (a) Depth-first search (b) Breadth-first search (c) Unifrm-cst search (d) Greedy search (e) A* search 3 5 19 B h=14 C D 6 h=18 4 5 5 h=15 E F G H A h=10 h=12 h=8 h=10 h=20 CS 460, Sessins 8-9 8
Breadth-first search Nde queue: initializatin # state depth path cst parent # 1 A 0 0 -- CS 460, Sessins 8-9 9
Breadth-first search Nde queue: add successrs t queue end; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 CS 460, Sessins 8-9 10
Breadth-first search Nde queue: add successrs t queue end; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 CS 460, Sessins 8-9 11
Breadth-first search Nde queue: add successrs t queue end; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 CS 460, Sessins 8-9 12
Eercise: Search Algrithms The fllwing figure shws a prtin f a partially epanded search tree. Each arc between ndes is labeled with the cst f the crrespnding peratr, and the leaves are labeled with the value f the heuristic functin, h. Which nde (use the nde s letter) will be epanded net by each f the fllwing search algrithms? (a) Depth-first search (b) Breadth-first search (c) Unifrm-cst search (d) Greedy search (e) A* search 3 5 19 B h=14 C D 6 h=18 4 5 5 h=15 E F G H A h=10 h=12 h=8 h=10 h=20 CS 460, Sessins 8-9 13
Unifrm-cst search Nde queue: initializatin # state depth path cst parent # 1 A 0 0 -- CS 460, Sessins 8-9 14
Unifrm-cst search Nde queue: add successrs t queue s that entire queue is srted by path cst s far; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 D 1 5 1 4 C 1 19 1 CS 460, Sessins 8-9 15
Unifrm-cst search Nde queue: add successrs t queue s that entire queue is srted by path cst s far; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 4 C 1 19 1 CS 460, Sessins 8-9 16
Unifrm-cst search Nde queue: add successrs t queue s that entire queue is srted by path cst s far; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 4 C 1 19 1 CS 460, Sessins 8-9 17
Eercise: Search Algrithms The fllwing figure shws a prtin f a partially epanded search tree. Each arc between ndes is labeled with the cst f the crrespnding peratr, and the leaves are labeled with the value f the heuristic functin, h. Which nde (use the nde s letter) will be epanded net by each f the fllwing search algrithms? (a) Depth-first search (b) Breadth-first search (c) Unifrm-cst search (d) Greedy search (e) A* search 3 5 19 B h=14 C D 6 h=18 4 5 5 h=15 E F G H A h=10 h=12 h=8 h=10 h=20 CS 460, Sessins 8-9 18
Greedy search Nde queue: initializatin # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- CS 460, Sessins 8-9 19
Greedy search Nde queue: Add successrs t queue, srted by cst t gal. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 3 D 1 5 15 20 1 4 C 1 19 18 37 1 Srt key CS 460, Sessins 8-9 20
Greedy search Nde queue: Add successrs t queue, srted by cst t gal. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 7 E 2 7 10 17 2 6 H 2 9 10 19 2 8 F 2 8 12 20 2 3 D 1 5 15 20 1 4 C 1 19 18 37 1 CS 460, Sessins 8-9 21
Greedy search Nde queue: Add successrs t queue, srted by cst t gal. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 7 E 2 7 10 17 2 6 H 2 9 10 19 2 8 F 2 8 12 20 2 3 D 1 5 15 20 1 4 C 1 19 18 37 1 CS 460, Sessins 8-9 22
Eercise: Search Algrithms The fllwing figure shws a prtin f a partially epanded search tree. Each arc between ndes is labeled with the cst f the crrespnding peratr, and the leaves are labeled with the value f the heuristic functin, h. Which nde (use the nde s letter) will be epanded net by each f the fllwing search algrithms? (a) Depth-first search (b) Breadth-first search (c) Unifrm-cst search (d) Greedy search (e) A* search 3 5 19 B h=14 C D 6 h=18 4 5 5 h=15 E F G H A h=10 h=12 h=8 h=10 h=20 CS 460, Sessins 8-9 23
A* search Nde queue: initializatin # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- CS 460, Sessins 8-9 24
A* search Nde queue: Add successrs t queue, srted by ttal cst. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 3 D 1 5 15 20 1 4 C 1 19 18 37 1 Srt key CS 460, Sessins 8-9 25
A* search Nde queue: Add successrs t queue frnt, srted by ttal cst. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 6 E 2 7 10 17 2 7 H 2 9 10 19 2 3 D 1 5 15 20 1 8 F 2 8 12 20 2 4 C 1 19 18 37 1 CS 460, Sessins 8-9 26
A* search Nde queue: Add successrs t queue frnt, srted by ttal cst. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 6 E 2 7 10 17 2 7 H 2 9 10 19 2 3 D 1 5 15 20 1 8 F 2 8 12 20 2 4 C 1 19 18 37 1 CS 460, Sessins 8-9 27
Eercise: Search Algrithms The fllwing figure shws a prtin f a partially epanded search tree. Each arc between ndes is labeled with the cst f the crrespnding peratr, and the leaves are labeled with the value f the heuristic functin, h. Which nde (use the nde s letter) will be epanded net by each f the fllwing search algrithms? (a) Depth-first search (b) Breadth-first search (c) Unifrm-cst search (d) Greedy search (e) A* search 3 5 19 B h=14 C D 6 h=18 4 5 5 h=15 E F G H A h=10 h=12 h=8 h=10 h=20 CS 460, Sessins 8-9 28
Last time: Simulated annealing algrithm Idea: Escape lcal etrema by allwing bad mves, but gradually decrease their size and frequency. - Nte: gal here is t maimize E. CS 460, Sessins 8-9 29
Last time: Simulated annealing algrithm Idea: Escape lcal etrema by allwing bad mves, but gradually decrease their size and frequency. < - - Algrithm when gal is t minimize E. CS 460, Sessins 8-9 30
This time: Outline Game playing The minima algrithm Resurce limitatins alpha-beta pruning Elements f chance CS 460, Sessins 8-9 31
What kind f games? Abstractin: T describe a game we must capture every relevant aspect f the game. Such as: Chess Tic-tac-te Accessible envirnments: Such games are characterized by perfect infrmatin Search: game-playing then cnsists f a search thrugh pssible game psitins Unpredictable ppnent: intrduces uncertainty thus game-playing must deal with cntingency prblems CS 460, Sessins 8-9 32
Searching fr the net mve Cmpleity: many games have a huge search space Chess: b = 35, m=100 ndes = 35 100 if each nde takes abut 1 ns t eplre then each mve will take abut 10 50 millennia t calculate. Resurce (e.g., time, memry) limit: ptimal slutin nt feasible/pssible, thus must apprimate 1. Pruning: makes the search mre efficient by discarding prtins f the search tree that cannt imprve quality result. 2. Evaluatin functins: heuristics t evaluate utility f a state withut ehaustive search. CS 460, Sessins 8-9 33
Tw-player games A game frmulated as a search prblem: Initial state:? Operatrs:? Terminal state:? Utility functin:? CS 460, Sessins 8-9 34
Tw-player games A game frmulated as a search prblem: Initial state: Operatrs: Terminal state: Utility functin: bard psitin and turn definitin f legal mves cnditins fr when game is ver a numeric value that describes the utcme f the game. E.g., -1, 0, 1 fr lss, draw, win. (AKA payff functin) CS 460, Sessins 8-9 35
Game vs. search prblem CS 460, Sessins 8-9 36
Eample: Tic-Tac-Te CS 460, Sessins 8-9 37
Type f games CS 460, Sessins 8-9 38
Type f games CS 460, Sessins 8-9 39
The minima algrithm Perfect play fr deterministic envirnments with perfect infrmatin Basic idea: chse mve with highest minima value = best achievable payff against best play Algrithm: 1. Generate game tree cmpletely 2. Determine utility f each terminal state 3. Prpagate the utility values upward in the three by applying MIN and MAX peratrs n the ndes in the current level 4. At the rt nde use minima decisin t select the mve with the ma (f the min) utility value Steps 2 and 3 in the algrithm assume that the ppnent will play perfectly. CS 460, Sessins 8-9 40
Generate Game Tree CS 460, Sessins 8-9 41
Generate Game Tree CS 460, Sessins 8-9 42
Generate Game Tree CS 460, Sessins 8-9 43
Generate Game Tree 1 ply 1 mve CS 460, Sessins 8-9 44
CS 460, Sessins 8-9 45 A subtree win lse draw
CS 460, Sessins 8-9 46 What is a gd mve? win lse draw
Minima 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 47
Minima MIN 3 2 2 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 48
Minima MAX MIN 3 3 2 2 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 49
Minima MAX MIN 3 3 2 2 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 50
minima = maimum f the minimum 1 st ply 2 nd ply CS 460, Sessins 8-9 51
Minima: Recursive implementatin Cmplete:? Optimal:? Time cmpleity:? Space cmpleity:? CS 460, Sessins 8-9 52
Minima: Recursive implementatin Cmplete: Yes, fr finite state-space Optimal: Yes Time cmpleity: O(b m ) Space cmpleity: O(bm) (= DFS Des nt keep all ndes in memry.) CS 460, Sessins 8-9 53
D We Have T D All That Wrk? MAX MIN 3 12 8 CS 460, Sessins 8-9 54
D We Have T D All That Wrk? MAX 3 MIN 3 3 12 8 CS 460, Sessins 8-9 55
D We Have T D All That Wrk? MAX MIN 3 3 12 8 2 3 2 Since 2 is smaller than 3, then there is n need fr further search CS 460, Sessins 8-9 56
D We Have T D All That Wrk? MAX MIN 3 3 12 8 3 X 2 14 5 2 Mre n this net time: α-β pruning CS 460, Sessins 8-9 57
1. Mve evaluatin withut cmplete search Cmplete search is t cmple and impractical Evaluatin functin: evaluates value f state using heuristics and cuts ff search New MINIMAX: CUTOFF-TEST: cutff test t replace the terminatin cnditin (e.g., deadline, depth-limit, etc.) EVAL: evaluatin functin t replace utility functin (e.g., number f chess pieces taken) CS 460, Sessins 8-9 58
Evaluatin functins Weighted linear evaluatin functin: t cmbine n heuristics f = w 1 f 1 + w 2 f 2 + + w n f n E.g, w s culd be the values f pieces (1 fr prawn, 3 fr bishp etc.) f s culd be the number f type f pieces n the bard CS 460, Sessins 8-9 59
Nte: eact values d nt matter CS 460, Sessins 8-9 60
Minima with cutff: viable algrithm? Assume we have 100 secnds, evaluate 10 4 ndes/s; can evaluate 10 6 ndes/mve CS 460, Sessins 8-9 61
2. α-β pruning: search cutff Pruning: eliminating a branch f the search tree frm cnsideratin withut ehaustive eaminatin f each nde α-β pruning: the basic idea is t prune prtins f the search tree that cannt imprve the utility value f the ma r min nde, by just cnsidering the values f ndes seen s far. Des it wrk? Yes, in rughly cuts the branching factr frm b t b resulting in duble as far lk-ahead than pure minima CS 460, Sessins 8-9 62
α-β pruning: eample MAX 6 MIN 6 6 12 8 CS 460, Sessins 8-9 63
α-β pruning: eample MAX 6 MIN 6 2 6 12 8 2 CS 460, Sessins 8-9 64
α-β pruning: eample MAX 6 MIN 6 2 5 6 12 8 2 5 CS 460, Sessins 8-9 65
α-β pruning: eample MAX 6 Selected mve MIN 6 2 5 6 12 8 2 5 CS 460, Sessins 8-9 66
α-β pruning: general principle Player Oppnent m α If α > v then MAX will chse m s prune tree under n Player Similar fr β fr MIN Oppnent n v CS 460, Sessins 8-9 67
Prperties f α-β CS 460, Sessins 8-9 68
The α-β algrithm: CS 460, Sessins 8-9 69
Mre n the α-β algrithm Same basic idea as minima, but prune (cut away) branches f the tree that we knw will nt cntain the slutin. CS 460, Sessins 8-9 70
Mre n the α-β algrithm: start frm Minima CS 460, Sessins 8-9 71
Remember: Minima: Recursive implementatin Cmplete: Yes, fr finite state-space Optimal: Yes Time cmpleity: O(b m ) Space cmpleity: O(bm) (= DFS Des nt keep all ndes in memry.) CS 460, Sessins 8-9 72
Mre n the α-β algrithm Same basic idea as minima, but prune (cut away) branches f the tree that we knw will nt cntain the slutin. Because minima is depth-first, let s cnsider ndes alng a given path in the tree. Then, as we g alng this path, we keep track f: α : Best chice s far fr MAX β : Best chice s far fr MIN CS 460, Sessins 8-9 73
Mre n the α-β algrithm: start frm Minima Nte: These are bth Lcal variables. At the Start f the algrithm, We initialize them t α = - and β = + CS 460, Sessins 8-9 74
Mre n the α-β algrithm In Min-Value: MAX α = - β = + MIN Ma-Value lps ver these MAX Min-Value lps ver these 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 CS 460, Sessins 8-9 75
Mre n the α-β algrithm In Ma-Value: MAX α = - β = + MIN Ma-Value lps ver these α = 5 β = + MAX 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 CS 460, Sessins 8-9 76
Mre n the α-β algrithm In Min-Value: MAX MIN α = 5 β = + α = - β = + MAX Min-Value lps ver these 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 α = 5 β = 2 End lp and return 5 CS 460, Sessins 8-9 77
Mre n the α-β algrithm In Ma-Value: MAX α = - β = + MIN Ma-Value lps ver these α = 5 β = + α = 5 β = + MAX 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 α = 5 β = 2 End lp and return 5 CS 460, Sessins 8-9 78
Anther way t understand the algrithm Frm: http://yda.cis.temple.edu:8080/ugaiwww/lectures95/search/alpha-beta.html Fr a given nde N, α is the value f N t MAX β is the value f N t MIN CS 460, Sessins 8-9 79
Eample CS 460, Sessins 8-9 80
α-β algrithm: CS 460, Sessins 8-9 81
Slutin NODE TYPE ALPHA BETA SCORE A Ma -I +I B Min -I +I C Ma -I +I D Min -I +I E Ma 10 10 10 D Min -I 10 F Ma 11 11 11 D Min -I 10 10 C Ma 10 +I G Min 10 +I H Ma 9 9 9 G Min 10 9 9 C Ma 10 +I 10 B Min -I 10 J Ma -I 10 K Min -I 10 L Ma 14 14 14 K Min -I 10 10 NODE TYPE ALPHA BETA SCORE J Ma 10 10 10 B Min -I 10 10 A Ma 10 +I Q Min 10 +I R Ma 10 +I S Min 10 +I T Ma 5 5 5 S Min 10 5 5 R Ma 10 +I V Min 10 +I W Ma 4 4 4 V Min 10 4 4 R Ma 10 +I 10 Q Min 10 10 10 A Ma 10 10 10 CS 460, Sessins 8-9 82
State-f-the-art fr deterministic games CS 460, Sessins 8-9 83
Nndeterministic games CS 460, Sessins 8-9 84
Algrithm fr nndeterministic games CS 460, Sessins 8-9 85
Remember: Minima algrithm CS 460, Sessins 8-9 86
Nndeterministic games: the element f chance epectima and epectimin, epected values ver all pssible utcmes CHANCE? 0.5 0.5 3?? 8 17 8 CS 460, Sessins 8-9 87
Nndeterministic games: the element f chance Epectima CHANCE 4 = 0.5*3 + 0.5*5 0.5 0.5 3 5 Epectimin 5 8 17 8 CS 460, Sessins 8-9 88
Evaluatin functins: Eact values DO matter Order-preserving transfrmatin d nt necessarily behave the same! CS 460, Sessins 8-9 89
State-f-the-art fr nndeterministic games CS 460, Sessins 8-9 90
Summary CS 460, Sessins 8-9 91
Eercise: Game Playing Cnsider the fllwing game tree in which the evaluatin functin values are shwn belw each leaf nde. Assume that the rt nde crrespnds t the maimizing player. Assume the search always visits children left-t-right. (a) Cmpute the backed-up values cmputed by the minima algrithm. Shw yur answer by writing values at the apprpriate ndes in the abve tree. (b) Cmpute the backed-up values cmputed by the alpha-beta algrithm. What ndes will nt be eamined by the alpha-beta pruning algrithm? (c) What mve shuld Ma chse nce the values have been backed-up all the way? B E F G H I J K C L M N O P Q R S T U V W X Y 2 3 8 5 7 6 0 1 5 2 8 4 10 2 A D Ma Min Ma Min CS 460, Sessins 8-9 92