Final Exam V1. CS 188 Fall Introduction to Artificial Intelligence

Transcription

1 CS 88 Fall 7 Introduction to Artificial Intelligence Final Eam V You have approimatel 7 minutes. The eam is closed book, closed calculator, and closed notes ecept our three-page crib sheet. Mark our answers ON THE EXAM ITSELF. If ou are not sure of our answer ou ma wish to provide a brief eplanation. All short answer sections can be successfull answered in a few sentences AT MOST. For multiple choice questions: means mark all options that appl means mark a single choice When selecting an answer, please fill in the bubble or square completel ( and ) First name Last name SID Student to our right Student to our left Your Discussion/Eam Prep* TA (fill all that appl): Brijen (Tu) Peter (Tu) David (Tu) Nipun (Tu) Wenjing (Tu) Aaron (W) Mitchell (W) Abhishek (W) Carn (W) Anwar (W) Aarash (W) Daniel (W) Yuchen* (Tu) And* (Tu) Nikita* (Tu) Shea* (W) Daniel* (W) For staff use onl: Q. Agent Testing Toda! / Q. Potpourri /4 Q. Search /9 Q4. CSPs /8 Q5. Game Trees /9 Q6. Something Fish / Q7. Polic Evaluation /8 Q8. Baes Nets: Inference /8 Q9. Decision Networks and VPI /9 Q. Neural Networks: Representation /5 Q. Backpropagation /9 Total /

2 THIS PAGE IS INTENTIONALLY LEFT BLANK

3 Q. [ pt] Agent Testing Toda! SID: It s testing time! Not onl for ou, but for our CS88 robots as well! Circle our favorite robot below.

4 Q. [4 pts] Potpourri (a) [ pt] Fill in the unlabelled nodes in the Baes Net below with the variables {A, B, C, E} such that the following independence assertions are true:. A B E, D. E D B. E C A, B 4. C D A, B D (b) [4 pts] For each of the 4 plots below, create a classification dataset which can or cannot be classified correctl b Naive Baes and perceptron, as specified. Each dataset should consist of nine points represented b the boes, shading the bo for positive class or leaving it blank for negative class. Mark Not Possible if no such dataset is possible. For can be classified b Naive Baes, there should be some probabilit distributions P (Y ) and P (F Y ), P (F Y ) for the class Y and features F, F that can correctl classif the data according to the Naive Baes rule, and for cannot there should be no such distribution. For perceptron, assume that there is a bias feature in addition to F and F. Naive Baes and perceptron both can classif: Naive Baes and perceptron both cannot classif: F F Not Possible Naive Baes can classif; perceptron cannot classif: F F Not Possible F F Not Possible Naive Baes cannot classif; perceptron can classif: (c) [ pt] Consider a multi-class perceptron for classes A, B, and C with current weight vectors: w A = (, 4, 7), w B = (,, 6), w C = (7, 9, ) F F Not Possible A new training sample is now considered, which has feature vector f() = (,, ) and label = B. What are the resulting weight vectors after the perceptron has seen this eample and updated the weights? w A = w B = w C =

5 SID: (d) [ pt] A single perceptron can compute the XOR function. True False (e) [ pt] A perceptron is guaranteed to learn a separating decision boundar for a separable dataset within a finite number of training steps. True False (f) [ pt] Given a linearl separable dataset, the perceptron algorithm is guaranteed to find a ma-margin separating hperplane. True False (g) [ pt] You would like to train a neural network to classif digits. Your network takes as input an image and outputs probabilities for each of the classes, -9. The network s prediction is the class that it assigns the highest probabilit to. From the following functions, select all that would be suitable loss functions to minimize using gradient descent: The square of the difference between the correct digit and the digit predicted b our network The probabilit of the correct digit under our network The negative log-probabilit of the correct digit under our network None of the above (h) [ pt] From the list below, mark all triples that are inactive. A shaded circle means that node is conditioned on. (i) [ pts] A Consider the gridworld above. At each timestep the agent will have two available actions from the set {North, South, East, W est}. Actions that would move the agent into the wall ma never be chosen, and allowed actions alwas succeed. The agent receives a reward of +8 ever time it enters the square marked A. Let the discount factor be γ =. At each cell in the following tables, fill in the value of that state after iteration k of Value Iteration. k = k = k = k = (j) [ pt] Consider an HMM with T timesteps, hidden state variables X,... X T, and observed variables E,... E T. Let S be the number of possible states for each hidden state variable X. We want to compute (with the forward algorithm) or estimate (with particle filtering) P (X T E = e,... E T = e T ). How man particles, in terms of S and T, would it take for particle filtering to have the same time compleit as the forward algorithm? You can assume that, in particle filtering, each sampling step can be done in constant time for a single particle (though this is not necessaril the case in realit): # Particles =

6 Q. [9 pts] Search Suppose we have a connected graph with N nodes, where N is finite but large. Assume that ever node in the graph has eactl D neighbors. All edges are undirected. We have eactl one start node, S, and eactl one goal node, G. Suppose we know that the shortest path in the graph from S to G has length L. That is, it takes at least L edge-traversals to get from S to G or from G to S (and perhaps there are other, longer paths). We ll consider various algorithms for searching for paths from S to G. (a) [ pts] Uninformed Search Using the information above, give the tightest possible bounds, using big O notation, on both the absolute best case and the absolute worst case number of node epansions for each algorithm. Your answer should be a function in terms of variables from the set {N, D, L}. You ma not need to use ever variable. (i) [ pt] DFS Graph Search Best case: Worst case: (ii) [ pt] BFS Tree Search Best case: Worst case: (b) [ pts] Bidirectional Search Notice that because the graph is undirected, finding a path from S to G is equivalent to finding a path from G to S, since reversing a path gives us a path from the other direction of the same length. This fact inspired bidirectional search. As the name implies, bidirectional search consists of two simultaneous searches which both use the same algorithm; one from S towards G, and another from G towards S. When these searches meet in the middle, the can construct a path from S to G. More concretel, in bidirectional search: We start Search from S and Search from G. The searches take turns popping nodes off of their separate fringes. First Search epands a node, then Search epands a node, then Search again, etc. This continues until one of the searches epands some node X which the other search has also epanded. At that point, Search knows a path from S to X, and Search knows a path from G to X, which provides us with a path from X to G. We concatenate those two paths and return our path from S to G. Don t stress about further implementation details here! Repeat part (a) with the bidirectional versions of the algorithms from before. Give the tightest possible bounds, using big O notation, on both the absolute best and worst case number of node epansions b the bidirectional search algorithm. Your bound should still be a function of variables from the set {N, D, L}. (i) [ pt] Bidirectional DFS Graph Search Best case: Worst case: (ii) [ pt] Bidirectional BFS Tree Search Best case: Worst case:

7 In parts (c)-(e) below, consider the following graph, with start state S and goal state G. Edge costs are labeled on the edges, and heuristic values are given b the h values net to each state. In the search procedures below, break an ties alphabeticall, so that if nodes on our fringe are tied in values, the state that comes first alphabeticall is epanded first. SID: h = h = S B A C 6 h = G h = 4 h = (c) [ pt] Greed Graph Search What is the path returned b greed graph search, using the given heuristic? S A G S A C G S B A C G S B A G S B C G (d) A* Graph Search (i) [ pt] List the nodes in the order the are epanded b A* graph search: Order: (ii) [ pt] What is the path returned b A* graph search? S A G S A C G S B A C G S B A G S B C G (e) Heuristic Properties (i) [ pt] Is this heuristic admissible? If so, mark Alread admissible. If not, find a minimal set of nodes that would need to have their values changed to make the heuristic admissible, and mark them below. Alread admissible Change h(s) Change h(a) Change h(b) Change h(c) Change h(d) Change h(g) (ii) [ pt] Is this heuristic consistent? If so, mark Alread consistent. If not, find the minimal set of nodes that would need to have their values changed to make the heuristic consistent, and mark them below. Alread consistent Change h(s) Change h(a) Change h(b) Change h(c) Change h(d) Change h(g)

8 Q4. [8 pts] CSPs Four people, A, B, C, and D, are all looking to rent space in an apartment building. There are three floors in the building,,, and (where is the lowest floor and is the highest). Each person must be assigned to some floor, but it s ok if more than one person is living on a floor. We have the following constraints on assignments: A and B must not live together on the same floor. If A and C live on the same floor, the must both be living on floor. If A and C live on different floors, one of them must be living on floor. D must not live on the same floor as anone else. D must live on a higher floor than C. We will formulate this as a CSP, where each person has a variable and the variable values are floors. (a) [ pt] Draw the edges for the constraint graph representing this problem. Use binar constraints onl. You do not need to label the edges. A B C D (b) [ pts] Suppose we have assigned C =. Appl forward checking to the CSP, filling in the boes net to the values for each variable that are eliminated: A B C D (c) [ pts] Starting from the original CSP with full domains (i.e. without assigning an variables or doing the forward checking in the previous part), enforce arc consistenc for the entire CSP graph, filling in the boes net to the values that are eliminated for each variable: A B C D (d) [ pts] Suppose that we were running local search with the min-conflicts algorithm for this CSP, and currentl have the following variable assignments. A B C D Which variable would be reassigned, and which value would it be reassigned to? Assume that an ties are broken alphabeticall for variables and in numerical order for values. The variable A will be assigned the new value B C D

9 Q5. [9 pts] Game Trees SID: The following problems are to test our knowledge of Game Trees. (a) Minima The first part is based upon the following tree. Upward triangle nodes are maimizer nodes and downward are minimizers. (small squares on edges will be used to mark pruned nodes in part (ii)) (i) [ pt] Complete the game tree shown above b filling in values on the maimizer and minimizer nodes. (ii) [ pts] Indicate which nodes can be pruned b marking the edge above each node that can be pruned (ou do not need to mark an edges below pruned nodes). In the case of ties, please prune an nodes that could not affect the root node s value. Fill in the bubble below if no nodes can be pruned. No nodes can be pruned

10 (b) Food Dimensions The following questions are completel unrelated to the above parts. Pacman is plaing a trick game. There are 4 portals to food dimensions. But, these portals are guarded b a ghost. Furthermore, neither Pacman nor the ghost know for sure how man pellets are behind each portal, though the know what options and probabilities there are for all but the last portal. Pacman moves first, either moving West or East. After which, the ghost can block of the portals available. You have the following gametree. The maimizer node is Pacman. The minimizer nodes are ghosts and the portals are chance nodes with the probabilities indicated on the edges to the food. In the event of a tie, the left action is taken. Assume Pacman and the ghosts pla optimall. West East P P P P X Y (i) [ pt] Fill in values for the nodes that do not depend on X and Y. (ii) [4 pts] What conditions must X and Y satisf for Pacman to move East? What about to definitel reach the P4? Keep in mind that X and Y denote numbers of food pellets and must be whole numbers: X, Y {,,,,... }. To move East: To reach P4:

11 Q6. [ pts] Something Fish SID: In this problem, we will consider the task of managing a fisher for an infinite number of das. (Fisheries farm fish, continuall harvesting and selling them.) Imagine that our fisher has a ver large, enclosed pool where we keep our fish. Harvest (pm): Before we go home each da at pm, we have the option to harvest some (possibl all) of the fish, thus removing those fish from the pool and earning us some profit, dollars for fish. Birth/death (midnight): At midnight each da, some fish are born and some die, so the number of fish in the pool changes. An ecologist has analzed the ecological dnamics of the fish population. The sa that if at midnight there are fish in the pool, then after midnight there will be eactl f() fish in the pool, where f is a function the have provided to us. (We will pretend it is possible to have fractional fish.) To ensure ou properl maimize our profit while managing the fisher, ou choose to model it using a Markov decision problem. For this problem we will define States and Actions as follows: State: the number of fish in the pool that da (before harvesting) Action: the number of fish ou harvest that da (a) [ pts] How will ou define the transition and reward functions? T (s, a, s ) = R(s, a) = (b) [4 pts] Suppose the discount rate is γ =.99 and f is as below. Graph the optimal polic π. # fish after midnight Fish population dnamic f # fish before midnight optimal action for da i (YOUR ANSWER) Optimal polic π state on da i (c) [4 pts] Suppose the discount rate is γ =.99 and f is as below. Graph the optimal polic π. # fish after midnight Fish population dnamic f # fish before midnight optimal action for da i (YOUR ANSWER) Optimal polic π state on da i

12 Q7. [8 pts] Polic Evaluation In this question, ou will be working in an MDP with states S, actions A, discount factor γ, transition function T, and reward function R. We have some fied polic π : S A, which returns an action a = π(s) for each state s S. We want to learn the Q function Q π (s, a) for this polic: the epected discounted reward from taking action a in state s and then continuing to act according to π: Q π (s, a) = s T (s, a, s )[R(s, a, s ) + γq π (s, π(s )]. The polic π will not change while running an of the algorithms below. (a) [ pt] Can we guarantee anthing about how the values Q π compare to the values Q for an optimal polic π? Q π (s, a) Q (s, a) for all s, a Q π (s, a) = Q (s, a) for all s, a Q π (s, a) Q (s, a) for all s, a None of the above are guaranteed (b) Suppose T and R are unknown. You will develop sample-based methods to estimate Q π. You obtain a series of samples (s, a, r ), (s, a, r ),... (s T, a T, r T ) from acting according to this polic (where a t = π(s t ), for all t). (i) [4 pts] Recall the update equation for the Temporal Difference algorithm, performed on each sample in sequence: V (s t ) ( α)v (s t ) + α(r t + γv (s t+ )) which approimates the epected discounted reward V π (s) for following polic π from each state s, for a learning rate α. Fill in the blank below to create a similar update equation which will approimate Q π using the samples. You can use an of the terms Q, s t, s t+, a t, a t+, r t, r t+, γ, α, π in our equation, as well as and ma with an inde variables (i.e. ou could write ma a, or a and then use a somewhere else), but no other terms. Q(s t, a t ) ( α)q(s t, a t ) + α [ ] (ii) [ pts] Now, we will approimate Q π using a linear function: Q(s, a) = w f(s, a) for a weight vector w and feature function f(s, a). To decouple this part from the previous part, use Q samp Q(s t, a t ) ( α)q(s t, a t ) + αq samp ). Which of the following is the correct sample-based update for w? w w + α[q(s t, a t ) Q samp ] w w α[q(s t, a t ) Q samp ] w w + α[q(s t, a t ) Q samp ]f(s t, a t ) w w α[q(s t, a t ) Q samp ]f(s t, a t ) w w + α[q(s t, a t ) Q samp ]w w w α[q(s t, a t ) Q samp ]w (iii) [ pt] The algorithms in the previous parts (part i and ii) are: model-based model-free for the value in the blank in part (i) (i.e.

13 Q8. [8 pts] Baes Nets: Inference SID: Consider the following Baes Net, where we have observed that D = +d. A B C D P (A) +a.5 a.5 P (B A) +a +b.5 +a b.5 a +b. a b.8 P (C A, B) +a +b +c.8 +a +b c. +a b +c.6 +a b c.4 a +b +c. a +b c.8 a b +c. a b c.9 P (D C) +c +d.4 +c d.6 c +d. c d.8 (a) [ pt] Below is a list of samples that were collected using prior sampling. Mark the samples that would be rejected b rejection sampling. +a b +c d +a b +c +d a +b c d +a +b +c +d (b) [ pts] To decouple from the previous part, ou now receive a new set of samples shown below: +a +b +c +d a b c +d +a +b +c +d +a b c +d a b c +d For this part, epress our answers as eact decimals or fractions simplified to lowest terms. Estimate the probabilit P (+a + d) if these new samples were collected using... (i) [ pt]... rejection sampling: (ii) [ pts]... likelihood weighting: (c) [4 pts] Instead of sampling, we now wish to use variable elimination to calculate P (+a + d). We start with the factorized representation of the joint probabilit: P (A, B, C, +d) = P (A)P (B A)P (C A, B)P (+d C) (i) [ pt] We begin b eliminating the variable B, which creates a new factor f. Complete the epression for the factor f in terms of other factors. f ( ) = (ii) [ pt] After eliminating B to create a factor f, we net eliminate C to create a factor f. What are the remaining factors after both B and C are eliminated? p(a) p(b A) p(c A, B) p(+d C) f f (iii) [ pts] After eliminating both B and C, we are now read to calculate P (+a + d). Write an epression for P (+a + d) in terms of the remaining factors. P (+a + d) =

14 Q9. [9 pts] Decision Networks and VPI (a) Consider the decision network structure given below: A M W U T S N Mark all of the following statements that could possibl be true, for some probabilit distributions for P (M), P (W ), P (T ), P (S M, W ), and P (N T, S) and some utilit function U(S, A): (i) [.5 pts] VPI(T ) < VPI(T ) = VPI(T ) > VPI(T ) = VPI(N) (ii) [.5 pts] VPI(T N) < VPI(T N) = VPI(T N) > VPI(T N) = VPI(T S) (iii) [.5 pts] VPI(M) > VPI(W ) VPI(M) > VPI(S) VPI(M) < VPI(S) VPI(M S) > VPI(S) (b) Consider the decision network structure given below. V A W U X Y Z Mark all of the following statements that are guaranteed to be true, regardless of the probabilit distributions for an of the chance nodes and regardless of the utilit function. (i) [.5 pts] VPI(Y ) = VPI(X) = VPI(Z) = VPI(W, Z) VPI(Y ) = VPI(Y, X) (ii) [.5 pts] VPI(X) VPI(W ) VPI(V ) VPI(W ) VPI(V W ) = VPI(V ) VPI(W V ) = VPI(W ) (iii) [.5 pts] VPI(X W ) = VPI(Z W ) = VPI(X, W ) = VPI(V, W ) VPI(W, Y ) = VPI(W ) + VPI(Y )

15 SID: Q. [5 pts] Neural Networks: Representation w w w w w w G : * * H : * * w b w w w b b w w G : * + * H : * + * w w w w w w G : * relu * H : * relu * w b w w w b b w w G 4 : * + relu * H 4 : * + relu * w b w b w w b b w w b G 5 : * + relu * + H 5 : * + relu * + For each of the piecewise-linear functions below, mark all networks from the list above that can represent the function eactl on the range (, ). In the networks above, relu denotes the element-wise ReLU nonlinearit: relu(z) = ma(, z). The networks G i use -dimensional laers, while the networks H i have some -dimensional intermediate laers. (a) [5 pts] G G G G 4 G 5 H H H H 4 H 5 None of the above G G G G 4 G 5 H H H H 4 H 5 None of the above (b) [5 pts] G G G G 4 G 5 H H H H 4 H 5 None of the above G G G G 4 G 5 H H H H 4 H 5 None of the above (c) [5 pts] G G G G 4 G 5 H H H H 4 H 5 None of the above G G G G 4 G 5 H H H H 4 H 5 None of the above

16 Q. [9 pts] Backpropagation In this question we will perform the backward pass algorithm on the formula [ ] A A Here, A =, = A A [ ], b = A = f = A [ ] [ A + A b ( ) = ], and f = A + A b b = b + b is a scalar. A b f (a) [ pt] Calculate the following partial derivatives of f. ] (i) [ pt] Find f b = [ ] [ f b f b. [ ] b b [ ] b b (b) [ pts] Calculate the following partial derivatives of b. ( b (i) [ pt] A, ) b A [ ] f(b ) f(b ) [ ] A A [ ] b + b b b (A, A ) (, ) (, ) (A, A ) (, ) ( (ii) [ pt] b A, ) b A (A, A ) (, ) (, ) (, ) (A, A ) (iii) [ pt] ( ) b, b (A, A ) (A, A ) (, ) (b, b ) (A, A ) (c) [ pts] Calculate the following partial derivatives of f. ( ) (i) [ pt] f f A, A (A, A ) (A b, A b ) (A, A ) ( b, b ) ( b, b ) ( b, b ) (ii) [ pt] ( f A, ) f A (A, A ) (A b, A b ) (A, A ) ( b, b ) ( b, b ) ( b, b ) (iii) [ pt] ( ) f, f (A b + A b, A b + A b ) (A b + A b, A b + A b ) (A b + A b, A b + A b ) (A b + A b, A b + A b ) (d) [ pts] Now we consider the general case where A is an n d matri, and is a d vector. As before, f = A. (i) [ pt] Find f A in terms of A and onl. A A A A ( A A ) (ii) [ pt] Find f in terms of A and onl. AA A ( A A ) A A A A A

17 THIS PAGE IS INTENTIONALLY LEFT BLANK SID: