Introduction. next up previous Next: Problem Description Up: Bayesian Networks Previous: Bayesian Networks

Transcription

1 Next: Problem Description Up: Bayesian Networks Previous: Bayesian Networks Introduction Autonomous agents often find themselves facing a great deal of uncertainty. It decides how to act based on its perceived state and that of the world, which is not necessarily the true state. An agent must rely on uncertain and incomplete data from its inputs. In other cases we may want the agent to predict future state which is fraught with even more uncertainty. If the agent's actions were based on probability distributions over states perhaps we can deal with this uncertainty effectively. Given this distribution over states and the (potentially probabilistic) model for the outcomes of its actions, it could compute the expected reward for an action or policy. To the degree that this information is accurate, the action or policy would be a good one. This chapter will focus on how to generate the probability distribution on which the agent will base its actions. In order to have the most accurate distribution, it must use all the information at its disposal. The Bayesian Network, sometimes referred to as a BayesNet, is a framework for using incorporating the available information into an estimate. Consider the car insurance agent. A client asks for a quote and the agent wants to provide the quote that maximizes expected profit for his company. His quote should be the one that maximizes This is very similar to a robot problem in which is the action model and is the state. The primary difficulty is that we don't know ahead of time whether or not this client will file any claims. If we had that information, the problem would be trivial. We could instead compute an expectation over claims, which serves to push the difficulty into estimating address.. This is precisely the problem I hope to

2 Given no information about the agent would be forced to resort to his prior distribution. That is likely the probability distribution of cost for all drivers. Then the expectation would tell him that this client would cost the average amount. However, the agent does have information about the client. Suppose that he knows that this client has been in 5 accidents in the last 7 years. A quote based on the average driver might not be prudent. How should the agent modify the quote? The agent now has some information on which to condition the probability. Rather than, he is now interested in. Suppose that the insurance company has compiled a database of all claims with client information for each. This allows the agent to quote the average among clients with 5 accidents, which is surely a better estimate. But that isn't all that the agent knows. He also knows the client is a male, age 22 who drives a 1995 Buick Regal. The only problem is that when we filter the claims database on {male,22,5-accidents, 1995 Buick Regal} we don't get any data back; this client would be the first. How can the agent use this information to improve his estimate? Next: Problem Description Up: Bayesian Networks Previous: Bayesian Networks root

3 Next: Java API Up: Problem Description Previous: Problem Description Subsections Inputs Correctness Restrictions Problem Statement The goal is to rationally estimate probability using all the information available. To do this we will generate a BayesNet. A BayesNet is a directed-acyclic graph whose nodes denote variables and whose edges denote ``relationships.'' A node contains the conditional probability distribution of its variable conditioned on its parents in the graph. Therefore, if two nodes are connected by an edge, the variables are dependant. The converse, however is not true. Suppose that I'm interested in the probability of the grass being wet and I know that it is both cloudy and raining (fig 2.1). The sky being cloudy and the grass being wet are dependant. But knowing whether or

4 not it is raining makes the cloudy information irrelevant. There is an edge between two nodes only they are dependant regardless of the other information available. This is the essential feature of a Bayesian Network, i.e. some variables are not connected by edges because they are conditionally independent. This is a key insight that dramatically simplifies the model. Finally, we define the direction of the edges. We can consider these to be directions of causality. That, however, is ambiguous. In figure 2.1 the causality is clear but that is not the case in general. Consider the relationship between income and level of education for instance. It turns out that if we are only considering two variables, the direction of the edge does not matter. In fact, if it did that would suggest a serious flaw. But between three or more variables, it does matter. Figure 2.1 shows the difference between a chain of causality and root causality between three variables. Suppose I go to school on weekdays and I have ballet on Tuesday, Thursday and Sunday. Given the day of the week, school and ballet are independent. Figure 2.1a is correct. Figure 2.1b is not correct because if we knew that it was a school day, ballet is not independent of the day of the week i.e. if we then learned the day of the week, the probability of ballet would change. Inputs Figure 1:figure 2 There are three scenarios in which BayesNets are typically used. They are used when we start out knowing either

5 the variables, their relationships and their joint distributions, variables and their relationships or, just the variables. These cases are progressively more difficult because each includes the former as a sub-problem. This chapter will focus on the first case. The second and third cases are essentially supervised learning problems and are beyond the current scope. 1 Given a complete BayesNet we can take values of a subset of the variables and generate a probability distribution over a set of the remaining variables. Given the structure of a BayesNet and sample data over its variables, a simple way of generating the Correctness Given a complete BayesNet of modest size or convenient structure, it is possible to compute exact probabilities, that is probabilities that are consistent with Bayes' rule. In this case, that will constitute correctness. For larger networks estimates can be achieved with sampling. The algorithm that I will present here is an exact inference algorithm. Restrictions In this chapter I will only consider discrete variables. This will make the joint distributions into tables rather than multivariable functions which will simplify the treatment. Since continuous variables can be discretized, the convenience comes at only a small loss of expressiveness. However, we shall see that large variable domains has a tremendous impact on computational expense. Next: Java API Up: Problem Description Previous: Problem Description root

6 Next: Method Description Up: Problem Description Previous: Java API Example Usage Given this API, the car insurance agent would be able to find out the probability distribution over the cost of insuring the client in front of him. Suppose that someone in company headquarters had already generated a complete Bayesian Network and stored the data in the file insurance.bn. The agent could instantiated the network with BayesNet bn = XMLBayesNet.readGraph(``insurance.bn''); Had company headquarters only supplied the structure but not the probabilities, the agent could have created the as above and then initialized it with bn = CPTEstimator.estimateCPTs(``claims.data'');. With the BayesNet instantiated, he could query it as follows. Assignment clientdata = XMLBayesNet.readAssignment(``client.data''); Factor cost = bn.query(``cost'',clientdata); System.out.println(cost); The above functions use XML-style files as input. The formats of those files are defined here. Here are example files for a network, assignment and data set. root

7 Next: Variable Elimination Up: Bayesian Networks Previous: Example Usage Method Description In the inference case, we are supplied with a Bayesian Network and we want to be able to answer questions of the form, ``What is where is the name of a variable in the network and is a partial assignment to values of other variables in the network. Let denote the remaining unassigned variables. To determine the probability over the query variable, we can sum out all the unassigned variables. If we were to do this directly, the BayesNet would offer us no advantage since we would need to know the whole joint distribution over. But from the structure of the network, we can decouple the distribution. For instance in the network in figure 3 and a query on variable we know that, where is a normalization constant. Suppose that we are given but none of the other variables. But is one since it is known. Therefore

8 This makes the original expression simply which is much more compact. Figure 2: Subsections Variable Elimination Example run Next: Variable Elimination Up: Bayesian Networks Previous: Example Usage root

9 Next: Example run Up: Method Description Previous: Method Description Variable Elimination The inferrence algorithm I am implementing is called VARIABLEELIMINATION. It computes the joint distribution over the querry variable by eliminating all of the non-query variables as it combines them into the full joint distribution. Let me introduce the following terminology: a factor, is an very much like a joint probability distribution and is defined over a set of variables SCOPE( ). Since we are only dealing with discrete variables, a factor can be represented by probability table. A factor need not be normalized but its entries must sum to less than 1. This will make many operations simpler. To convert a factor into a joint distribution, we perform this normalization. The algorithm will proceed as follows. It maintains a set of factors initialized with the probability distributions supplied by the BayesNet. Any factor that contains evidence variables is adjusted to take that into account. In the discrete case that means selecting the appropriate row (or higher-dimensional equivalent) of the table. Once that initialization is complete, we iterate through all of the variables that we don't want and remove any mention of them from the factors. Delaying the details of this elimination for the moment, the next thing to is to consolidate all the remaining factors factor involving all the query variables. Factors are combined using the FACTORMULTIPLY operation. Finally normalizing this factor turns into a probability distribution. VARIABLEELIMINATION( set of querry variables, vector of evidence ( ), BayesNet, = <variables, edges, probability tables >) returns a probability distribution local set of factors, for all variables initially empty if else

10 endif endfor for all variables endfor local for all factors endfor local for all assignments to querry variables endfor return A single variable is eliminated from the factors by collecting all of the factors that reference it and combining them into a single factor and then by summing-out (marginalizing) the variable that we wished to eliminate. ELIMINATEVARIABLE variable to be eliminated, set of factors ) returns no longer containing local factor local set of factors for all if

11 else endif endfor return When we eliminate a variable, we add a new factor containing all the variables that appeared in a factor with the eliminated variable. This has the potential for causing a ballooning of the size of some tables. That can quickly become unmanageable. This effect is dependant on the order in which the variables are eliminated which I have left unspecified. It turns out that finding the best order is NP-hard. One heuristic solution is to eliminate the variables greedily i.e. always eliminate the one that creates the smallest new table. The function FACTORMULTIPLY combines two factors into a single equivalent factor over the union of the scopes. The new factor is the product of the original factors evaluated at the applicable subassignment. Recall that factors do not need to be normalized. FACTORMULTIPLY( factor, factor ) returns an equivalent factor, with local local local local for each assignment to endfor return

12 For example let, = A B C T T T.11 T T F.12 T F T.13 T F F.14 F T T.15 F T F.16 F F T.17 F F F.18 = B C D T T T.21 T T F.22 T F T.23 T F F.24 F T T.25 F T F.26 F F T.27 F F F.28 then FACTORMULTIPLY = A B C D

13 T T T T =.0231 T T T F =.0242 T T F T =.0276 T T F F =.0288 T F T T =.0325 T F T F =.0338 T F F T =.0378 T F F F =.0392 F T T T =.0315 F T T F =.0330 F T F T =.0368 F T F F =.0384 F F T T =.0425 F F T F =.0442 F F F T =.0486 F F F F =.0504 Now if we wanted to marginalize or sum-out a variable from a table, for example, from in the preceding example, we would use the MARGINALIZE function which takes a factor and a set of variables

14 and returns a new factor with those variables removed. Marginalization simply requires summing over the assignments to the variables that we wish to remove. MARGINALIZE( factor set of variables ) returns factor no longer including local set of variables for all assignments for all assignments endfor endfor return To remove from in the preceding example we would get = A C T T =.24 T F =.26 F T =.32 F F =.34 Next: Example run Up: Method Description Previous: Method Description root

15 Next: Demonstration Up: Method Description Previous: Variable Elimination Example run Let's consider a simplified version of our original motivating problem, the insurance network in figure 3.2. Each variable is from a binary domain. The query is on the ``cost'' variable, that is, ``What is the probability that there will be a claim?'' The evidence is that the client is a young male with a cheap car who lives in a shady neighborhood (see figure 3.2). We start by initializing with the tables for all the non-query, non-evidence nodes of the graph. But rather than use the entire tables for ``stolen'',``this car cost'' and ``safe driver'' we select for the information that we know. The factor table for ``safe driver'' becomes simply.3, the table for ``stolen'' becomes.05 and the table for ``this car cost'' becomes damage.1 no damage 0. Suppose that we decide to eliminate ``safe driver'' next. We see that the only variable that mentions it is ``accident.'' We multiply the ``safe driver'' and ``accident'' tables and get

16 safe/accident.03 unsafe/accident.14. When ``safe driver'' is marginalized this gives a probability of.17 of an accident. Let's next eliminate ``stolen.'' The only factor mentioning it is ``this car damage.'' When multiplied we get the new factor accident no accident stolen not stolen.95 0

17 which when ``stolen'' is marginalized becomes accident 1 no accident.05 Recall that the variable elimination order is not explicit. We are not restricted to work from top down, although sometimes it is perhaps more intuitive to do so. Next we might try to eliminate ``Accident'' but that would create a factor of size four. If we were to next eliminate either ``This car cost'' or ``Other car cost'' next, we would get a new table of size two. Choosing on this basis, is no guarantee of optimality (determining optimality is #p-hard), but in general offers a great deal of improvement. Next let's eliminate ``this car damage.'' I will not go through the details. The result is shown in figure 3.2. Next we choose to eliminate the ``crash'' node. This will create a compound node consisting of both ``this car cost'' and ``other car cost.'' as shown in figure 3.2.

18 crash T T T T F F F F other damage T T F F T T F F this cost T F T F T F T F Which marginalizes to other damage T T F F this cost T F T F Now we are left with only one two factors left, one of which is the query. So we have no choice but to eliminate the last remaining non-query factor. This yields other T T T T F F F F this T T F F T T F F cost T F T F T F T F

19 Which marginalizes to cost T F which is the answer to our query. Next: Demonstration Up: Method Description Previous: Variable Elimination root

20 Next: Larger Example Up: Demonstration Previous: Demonstration Performance Profile In order to quantify the limit of the ability of this algorithm, I generated a suite of random networks. I varied the number of variables between 4 and 64, the ratio of edges to nodes from 1 to 4, and the size of the domains (which are the same for all variables). For each network, I queried a single, randomly-chosen variable with no evidence. I repeated this three times for each network and recorded the average computation time. The results of these trials are represented in figures 3-6. Figures 3 and 4 are with a fixed domain size of 2 and 3 respectively. From these one can see how the computation time grows with the density of edges. Figures 5 and 6 are with fixed edge to variable ratio of 2 and 3 respectively. From these one can see how computation time grows with the size of the domains. As the domains get larger than about 6, the runs were terminated by memory constraints instead of time. And all of it looks like bad news for the practicality of this algorithm. This should not be surprising since in a multiply connected network, exact inference is NP-Complete 2. Figure 3:Computation time with domains of size 2

21 Figure 4:Computation time with domains of size 2

22 Figure 5:Computation time with edges/variable = 2

23 Figure 6:Computation time with edges/variable = 3 Next: Larger Example Up: Demonstration Previous: Demonstration root

24 Next: Package Loading Up: Demonstration Previous: Performance Profile Larger Example So one might ask how practical is this algorithm on real examples. The insurance network described in the previous section is actually a simplification of the larger network shown in Figure 7, which is commonly used in machine learning benchmarks. In addition to having about twice as many nodes, it is connected more tightly and the variable domains are considerably larger. The maximum domain size is about 8 values, while most have about 4. It takes about to perform Variable Elimination on this network. That speed is substantially better than what we would expect from a random network of similar size. Perhaps that owes itself to the fact the ``intuition'' involved in designing that network did a great deal to simplify it. This network is approaching the limit of the tractability of the algorithm while only skimming the surface of complexity of networks we might be interested in pursuing. This problem is shared by all exact solution methods, which, by the virtue of being exact, all entail a similar amount of computation. In harder practical cases, one must resort to inexact methods. Inexact methods aren't necessarily much of a drawback when you consider that the original probabilities themselves, on which the entire inference is based, are just estimates.

25 Figure 7: Next: Package Loading Up: Demonstration Previous: Performance Profile root

26 Up: Bayesian Networks Previous: Package Loading Bibliography 1 2 Daphne Koller and Nir Friedman. Bayesian Networks and Beyond. None, draft edition, Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, root