Causation as Production

Transcription

1 Causation as Production John Bell 1 Abstract. Recently I suggested that a cause is an event which, in its context of occurrence, is both necessary and sufficient for the effect. However this definition is only appropriate if there is a single potential cause of the effect. Consequently I suggest a generalization of the definition and discuss the resulting Production Theory. I suggest that this can be seen as a combination of a regularity theory in the Hume tradition and a dependence theory in the Lewis tradition, and argue that the Production Theory inherits the strengths of the component theories while avoiding their weaknesses. 1 INTRODUCTION In [1] I propose a formal, logico-pragmatic, theory of causation. This defines a cause to be an event which, in its context of occurrence, is both necessary and sufficient for the effect. However I now consider that this definition is only appropriate in cases of non-redundant causation, where there is only one potential cause for the effect. Consequently I suggest an appropriate generalization, by weakening the contextual necessity condition, and justify the resulting Production Theory by relating it to the philosophical tradition and to recent philosophical discussions of causation. The central theme of the discussion is Hall s [8, p. 225] suggestion that causation should be viewed as production: that event c is a cause of effect e if c helps to generate or bring about or produce e. However, Hall argues that production should be analyzed in terms of regularity and that it conflicts with dependence, whereas I argue that production involves a combination of (appropriate forms of) regularity and dependence. In 2 I briefly recall the regularity theories of Hume and Mill, provide an informal introduction to the theory of sufficient causation (the contextual-sufficiency part of the Production Theory) in this setting, and discuss the major problem confronting regularity theories; that of distinguishing between regularities which are causal and regularities which are accidental. In 3 I recall Lewis s theory of causation as counterfactual dependence and argue that, like the contextual necessity condition that I gave in [1], the simple form of dependence he adopts is too restrictive because it excludes symmetrically redundant causes. Consequently I suggest a weaker form counterfactual dependence called individual dependence. Then, in the resulting Production Theory, a cause is defined to be an event which, in its context of occurrence, is both sufficient and individually necessary for the effect. The Production Theory can thus be regarded as a combination of a regularity theory and a dependence theory. In 4 I discuss Hall s [7] argument that transitivity and dependence conflict, and argue that they do so only if, as in Lewis s theory, depen- 1 Department of Computer Science, Queen Mary, University of London, United Kingdom, jb@dcs.qmul.ac.uk dence is taken to be a sufficient condition for causation. I conclude that, contrary to Hall s [p. 212] suggestion, dependence does bear a deep connection to causation, but it does so in conjunction with the theory of sufficient causation. Finally, in 5 I discuss the varieties of redundancy, and argue that, while these cause problems for simple dependence theories, they do not do so for the Production Theory. 2 REGULARITY Hume [10, Bk I, Pt III] suggests that we inductively acquire knowledge of regularities of succession of the form: A-type events are followed by B-type events. We then consider that A-type events cause B-type events, because whenever we see an A-type event, we expect it to be followed by a B-type event. Mill [15, Bk III, Ch 5] complicates this picture by considering assemblages of conditions. A single assemblage might consist of an A-type event together with certain conditions which must be present (positive conditions) and certain conditions which must be absent (negative conditions). For example, an assemblage concerning the lighting of matches might include the striking of the match, the presence of oxygen, and the absence of dampness in the match head. This idea is complicated by the possibility of a plurality of causes; the possibility that an effect can be realized in more than one way. For example, a match can also be lit (in the presence of oxygen and the absence of dampness, etc.) by touching its head with a red hot poker. So assemblages should be thought of as disjunctions of conjunctions of conditions. Mill suggests that it is possible, at least in principle, to specify assemblages which, if true, logically guarantee their associated effects. However, this does not appear to be a realistic possibility because in practice it is impossible to come up with an exhaustive list, especially of negative conditions. There is also the problem of interference from other events. Rather than begining with the concept of causation, I begin with a theory of events [4], in the AI reasoning-about-actions tradition [14, 16, 17, 18, 19], and use this as the basis for a definition of sufficient causation. A simplified version of the theory is described here. The full theory allows for the representation of context-dependent effects and non-deterministic effects; and hence of non-deterministic causation. The theory is interpreted in a possible partial worlds setting. Each possible partial world (henceforth simply world ) represents a possible history. At each world, time is taken to consist of points and to be discrete and linear. At any time point at a world, certain facts and certain event occurrences may be defined (be established as true or established as false). The idea behind the fact-event distinction is that events are dynamic (active) and introduce changes (impinge on the world, change the course of history), and facts are static (passive) and persist through time until affected by some event (until some event impinges on them). The partiality of worlds reflects notions such as

2 partial observability (we typically do not have complete knowledge of a world) and relevance/localness ( typically we are concerned only with a small region of it). These ideas can be conveyed graphically by means of the Lewistype neuron diagrams in Figure 1; reproduced from [8, Fig. 9.2]. In c d e 2 a b (a) c d 2 e 2 2 Figure 1. Two neuron diagrams: (a) c fires, (b) if c had not fired.... these diagrams, shaded vertices represent firing neurons, arrows represent stimulatory connections, and the remaining edges represent inhibitory connections. Thus, in (b) the effect of a firing is the subsequent firing of b. However, in (a), this effect is inhibited by the firing of c. Figures (a) and (b) can be thought of as representing alternative world histories in which different neurons fire in the same neuron network. World histories are described by the event language EL. For example, the occurs atom Occ(C)(1) states that neuron c fires at time 1 (at the world in question, at the actual world). The partial histories of worlds (a) and (b) can be represented by the following sets of atoms: w a = {Occ(C)(1), Occ(A)(1), Occ(D)(2), Occ(E)(3)}, w b = {Occ(A)(1), Occ(B)(2), Occ(E)(3)}. As non-firings have no effects they are, in the interests of economy, regarded as undefined occurrence atoms. The behaviour of the neurons can be defined by regarding neuron firings as event types, and by defining their preconditions, effects, and interactions. This is done by the axioms in Table 1. For example, Table 1. t(pre(c)(t) Occ(C)(t)) a b (b) Sample event definition axioms. t(eff(c)(t) (Occ(D)(t) TOcc(B)(t))) (1) t(pre(a)(t) Occ(A)(t)) t(eff(a)(t) Occ(B)(t)) tpref(c, A)(t) (2) t(pre(d)(t) Occ(D)(t)) t(eff(d)(t) Occ(E)(t)) (3) t(pre(b)(t) Occ(B)(t)) t(eff(b)(t) Occ(E)(t)) (4) Axiom (1) states that the precondition for c s firing is that c occurs (in effect no preconditions are required in this case), and its effects are that d fires and it is not the case that b fires. The operator T, used in this axiom, is a classically-valued (truth-value designation) operator: a sentence Tφ is true if φ is true, and is false otherwise (if φ is undefined or false). Consequently the combination T represents weak negation: the sentence Tφ is true if φ is false (if φ is true) or φ is undefined, and is false otherwise. The truth operator makes it possible to reason classically with partial information; in particular, when using weak negation only two cases need to be considered; either sentence φ is true or it is not. The final conjunct of Axiom (2) is explained below. Thus preconditions and effects embody specific causal knowledge. Their use is common in AI theories and has its origin in the representation used by the planner STRIPS [6]. Note the similarities with Hume s regularities and Mill s assemblages. Unlike most AI theories, the theory of events begins with the recognition that events are defeasible; that their preconditions are not always sufficient on occurrence for their effects. This may be because some unusual condition holds (the qualification problem), or because some other simultaneously occurring event interferes (the interaction problem), or, as Hume suggests, because we are mistaken in thinking that the regularity invariably holds (the problem of induction). In order to represent this, we begin with the notion of success. As Axiom (5) in Table 2 states, an event (token) succeeds (at time t) iff it is true that: the event occurs, its preconditions obtain, and its expected effects do indeed follow. The intention is that events Table 2. The Production Theory of causation. e, t(succ(e)(t) T(Occ(e)(t) Pre(e)(t) Eff(e)(t+1))) (5) R, x, t(inert(r, x)(t) (R(x)(t) R(x)(t+1))) (6) ɛ, φ, e, t(scause(ɛ, φ) ɛ, φ(cause 1 (ɛ, φ) (ɛ = Occ(e)(t) Succ(e)(t) ((θ Eff(e)(t+1)) φ) (θ φ))) (7) (SCause(ɛ, φ) ( Tɛ ( Tφ ɛ Cause 1 (ɛ, φ))))) (8) ɛ, ɛ, φ, ψ, χ(cause(ɛ, φ) (Cause 1 (ɛ, φ) (Cause 1 (ɛ, ɛ ) Cause(ɛ, φ)) (φ = (ψ χ) Cause(ɛ, ψ) Cause(ɛ, χ)))) (9) should normally succeed given their preconditions. So the preconditions should normally be sufficient on occurrence for the effect, and Axiom (5) should be used to infer by default that events succeed. So, given that the event occurs and its preconditions are true, Occ(e)(t) Pre(e)(t), the success assumption, Succ(e)(t), should, whenever consistent, be assumed, and the axiom used to infer its effects, Eff(e)(t+1). We can now return to the final conjunct of Axiom (2) which states that, in case of conflict, the success of event C is normally preferred to that of event A. Thus if, as in Figure 1(a), neurons c and a fire simultaneously, then the firing of c succeeds and the firing of a fails. More generally, event preferences can be used to represent the expected resolution of conflicts between events. Axiom schema (6) defines inertia, and states that the fact represented by relation R and variables x is inert at time t iff R(x)(t) and R(x)(t+1) have the same truth value. The axiom schema is intended to be used inductively to infer persistence whenever it is consistent to do so. The theory of events consists of axioms (5) and (6) given in Table 2, and an event theory is a theory containing these axioms. The intended interpretation of event theories is enforced by their formal pragmatics (their nonmonotonic semantics ) [3], which interprets event theories chronologically, giving priority to success assumptions over inertia assumptions in case of conflict and respecting the event preferences of the theory. The theory of causation is expressed in the more inclusive language CL; which admits quantification over EL sentences and which includes the modal operators and, which refer across worlds. Axiom (7) defines the notion of sufficient causation. It assumes a theory of background laws, which are represented as the conjunction θ and which include those defining the preconditions and effects of events. The axiom states that the occurrence of event e at time t is a sufficient cause (an SCause) of effect φ iff e succeeds at t, and φ is a logical consequence of the back-

3 ground laws together with e s defined effects but not of the background laws alone. For example, SCause(Occ(C)(1), Occ(D)(2)) and SCause(Occ(C(1), TOcc(B)(2)) are true at w a. The theory of sufficient causation can be viewed as a regularity theory in the Humeian tradition. It avoids the problems of traditional regularity theories raised by Lewis [11, pp ]; in particular, the problem of preemption is discussed in 5. However, the theory suffers from the fundamental problem of regularity theories, that of distinguishing between those regularities of succession which represent genuine causal connections and those which are merely accidental. In particular, the theory is faced with the inertia problem, as it is unable to distinguish between effects which are produced by events and those which remain true by inertia; for example [1, p. 2], whitewashing a white wall is sufficient, but not necessary, for the wall being white. The theory thus fails to capture an important feature of causes, namely that they are events which are efficacious, which make a difference, which alter the course of history, which impinge, which biff (to use a technical term of Lewis s [13, pp ] nontechnically). 3 DEPENDENCE Lewis [11, Ch. 21] goes further and equates causation with difference: We think of a cause as something that makes a difference, and the difference it makes must be a difference from what would have happened without it. Had it been absent, its effects some of them, at least, and usually all would have been absent as well [pp ]. Consequently Lewis suggests that causation should be understood in terms of counterfactual dependence between pairs of events. Thus, for distinct events c and e, the occurrence of e is counterfactually dependent on the occurrence of c iff (1) c and e both occur, and (2) if c had not occurred, then e would not have occurred. Dependence of this kind will be called simple dependence. The causation relation is then taken to be the ancestral (transitive closure) of the (counterfactual) dependence relation. Simple dependence can be represented in CL by a conditional of the form Tφ ψ, which, at a world at which φ is true, can, for present purposes, be understood as stating that ψ is true at the closest world at which Tφ is true; that removing φ from the world results in a world at which ψ is true. The formal pragmatics [3] ensures that the closest world to a given world is the expected one. (In general, there may be more than one closest world to a given world.) Thus, in the neuron example, both TOcc(C)(1) TOcc(D)(2) and TOcc(C)(1) Occ(B)(2) are true at w a (on the intended interpretation); as removing Occ(C)(1) from w a results in w b. Consequently, on the basis of simple dependence, Occ(C)(1) counts as the cause of both Occ(D)(2) and TOcc(B)(2) at w a. Hall argues that simple dependence is inadequate as a definition of causation, because it is neither a necessary [8, 3.2-3] nor a sufficient condition [7] for it. His argument against its sufficiency is discussed in the next section. His argument against its necessity can, I suggest, be made by appealing to cases of symmetrical redundancy. Redundancy (a phenomenon discussed more generally in 5) arises when two (or more) separate potential causes for a certain effect are present and either one by itself would have been followed by the effect, and so the question arises as to which, if any, of the events the effect depends on. The problem is particularly acute when the redundancy is symmetrical, when both candidates have equal claim to being called causes of the effect; as nothing, either obvious or hidden, breaks the tie between them. Lewis [12, p. 80] suggests that our intuitions in these cases are unclear. So that we may consider that neither event is a cause or consider that both events are causes; although we cannot reasonably say that one of the events is a cause and the other is not. The institution of execution by firing squad exploits this uncertainty. Supposing that the riflemen in the squad all aim true and fire simultaneously, then the victim s death is redundantly caused, and so each rifleman is tempted to think that his shot was not the fatal one. On Lewis s simple dependence theory there is no cause in such cases; because the effect does not depend on any one of the candidate causes. However, it seems more appropriate to say that each of the shots was a cause of the victim s death. This intuition is supported when causation is viewed as production. Each of the shots produces a noninertial effect (biffs), so it seems odd to say that the effect is uncaused simply because there is a surplus of (candidate) causes. Overproduction may remove simple dependence, but it is, nevertheless, still production. This accounts for the guilt which the members of the firing squad feel; each knows that his shot was sufficient to kill the victim and that it would have done so even if all of the others had missed. Consequently I suggest that simple dependence should be replaced by individual dependence as defined by Axiom (8) in Table 2. This states that event ɛ is a direct cause (a Cause 1) of effect φ iff ɛ is a sufficient cause of φ at a world and removing ɛ from the world results in a world at which either Tφ is true or some other event ɛ directly causes φ. The individual dependence condition thus ensures that every SCause in a given context biffs, but does not require that any of them biffs uniquely. For example, in the case of the neurons, Occ(C)(1) still counts as a cause of both Occ(D)(2) and TOcc(B)(2), by sufficiency and simple dependence, at w a. Moreover, we can obtain a formal analogue of the firing-squad example by supposing that (b) is changed so that d also fires, giving: w c = {Occ(A)(1), Occ(D)(2), Occ(B)(2), Occ(E)(3)}. Then Cause 1(Occ(D)(2), Occ(E)(3)) and Cause 1(Occ(B)(2), Occ(E)(3)) are both true at w c; as removing either cause results in a world at which the other causes e to fire (by sufficiency and simple dependence). The definition of Cause 1 is a generalization the definition of direct causation that I gave in [1]; which was restricted to cases of nonredundant or singular causation, and which can now be defined as follows: ɛ, φ(cause S(ɛ, φ) (Cause 1(ɛ, φ) ɛ ( ɛ = ɛ SCause(ɛ, φ)))). 4 TRANSITIVITY Transitivity is a property that is naturally associated with production when we think of processes or causal chains; for example, in Figure 1(a) it seems natural to conclude that the firing of c (indirectly) causes the firing of e. Accordingly, as stated by Axiom (9) in Table 2, the (indirect) causation relation Cause is taken to be transitive and to be closed under conjunction of effects. In this section three kinds of counterexample to transitivity are discussed. Counterexamples of the first kind raise what might be called the remoteness objection. It seems odd to say that Queen Victoria s birth caused her death. However, according to the Production Theory, the statement is true; Queen Victoria s birth initiated a causal process leading ultimately to the events which directly caused her death. The oddness of the claim arises because, while Queen Victoria s birth was indeed an indirect cause of her death, her birth is not adequate

4 as an explanation of her death. An adequate explanation would consist of causes which were more proximate to her death, and which contributed more directly to it. But determining which causes constitute an adequate explanation is a pragmatic problem, not a metaphysical one. The metaphysical task is to define the causal chain, the pragmatic task is to decide which parts of the chain are relevant to a particular explanation. An example of the second kind is Hall s [7, p. 187] engineer who: is standing by a switch in the railroad tracks. A train approaches in the distance. She flips the switch, so the train travels down the right-hand track instead of the left. Since the tracks reconverge up ahead, the train arrives at its destination all the same; let us further suppose that the time and manner of its arrival are exactly as they would have been had she not flipped the switch. Let c be the engineer s action and e be the train s arrival. Pick an event d that is part of the train s journey down the right-hand track. Clearly c is a cause of d and d of e; but is c a cause of e? Is her flipping the switch a cause of the train s arrival? Intuitively it might seem not, because it seems to be clear that the switching event makes no difference to whether the train arrives, but merely determines the route by which it arrives. However, Hall [ 4-5] argues that, despite this consideration, flipping the switch is a cause of the train s arrival. If, as the example persuades us to do, we think in terms of the train s arrival and dependence, then we are inclined to think that if the switch had not been flipped, the train would have arrived just the same, and so the switch cannot be considered to be a cause. However, if we think in terms of how the train s arrival was produced, then we need to consider the causal chain leading to it, and clearly this includes the switching event. In Hall s terminology, the switching event is a causal switch, as it changes the course of events; without it a different sequence of events would have produced (caused) the train s arrival. Readers who are unconvinced by this summary are referred to Hall s detailed and persuasive discussion. Formally, suppose an event theory analogous to the one for the neurons, which determines the following partial histories: w 0 = {Occ(C)(1), Occ(D)(2), Occ(E)(3)}, w 1 = {Occ(C)(1)}, w 2 = {Occ(B)(2), Occ(E)(3)}. Thus w 0 represents the actual world in which the switch is flipped, and the train travels via the right-hand track; w 1 represents the world where the switch is flipped, but the train s progress on the right-hand track is obstructed; and w 2 represents the world in which the switch is not flipped, and the train travels down the left-hand track. Then both Cause 1(Occ(C)(1), Occ(D)(2)) and Cause 1(Occ(D)(2), Occ(E)(3)) are true at w 0. Consequently Cause(Occ(C)(1), Occ(E)(3)) is true at w 0 by Axiom (9). Examples of the third kind involve cases of what Hall calls double prevention, and are designed to show that simple dependence cannot be a sufficient condition for causation if causation is taken to be transitive. In one of his examples [pp ]: Billy sees Suzy about to throw a water balloon at her neighbor s dog. He runs to try to stop her, but trips over a root and so fails. Suzy, totally oblivious to him, throws the water balloon at the dog.... Billy s running toward Suzy causes him to trip, which in turn causes the dog to yelp (by Dependence: if he hadn t tripped, he would have stopped her from throwing and so the dog wouldn t have yelped). But, intuitively, his running toward her does not cause the dog s yelp. In this example, Billy s trip is a double preventer because it prevents him from preventing her from throwing the balloon. Hall rightly suggests that double preventers are not causes and concludes [p. 182] that: the price of transitivity a price well worth paying is to give up the claim that there is any deep connection between counterfactual dependence and (the central kind of) causation. As there is no distinction between simple and individual dependence in examples like the one given, it seems that they challenge the necessity of individual dependence. However Hall s double-prevention examples all assume that dependence is a sufficient condition for causation (note, for instance, the appeal to dependence in the Billy-Suzy example), and so do not challenge the Production Theory because dependence is not taken to be a sufficient condition for causation; any Cause 1 is also required to be an SCause. 5 REDUNDANCY Symmetrical redundancy is discussed in 3, and it arises when two or more potential causes have equal claim to being regarded as the cause of an effect. Other redundancies are asymmetrical: intuition suggests that one of the events is a cause and the other is not; that one of the events is a preempting cause and that the other is a preempted cause. Lewis [12, p. 80] comments that: When our opinions are clear, it s incumbent on an analysis of causation to get them right. This turns out to be a severe test. Figure 1(a) provides an example of what Lewis calls early preemption. Both a and c fire simultaneously, with the firing of c causing (and thereby preempting the firing of a from causing) the firing of e. On Lewis s theory c counts as the cause because of the stepwise dependence of the firing of e on the firing of d, and the firing of d on the firing of c. In the case of the Production Theory, suppose the following partial histories: w 0 = {Occ(C)(1), Occ(A)(1), Occ(D)(2), Occ(E)(3)}, w 1 = {Occ(C)(1), Occ(A)(1)}, w 2 = {Occ(A)(1), Occ(B)(2), Occ(C)(3)}. Here w 0 represents the actual world, w 1 represents the alternative history in which the occurrence of D is removed, and and w 2 represents the alternative history in which the occurrence of C is removed. Then TOcc(C)(1) TOcc(D)(2) and TOcc(D)(2) TOcc(E)(3) are both true at w 0. So, Cause 1(Occ(C)(1), Occ(D)(2)) and Cause 1(Occ(D)(2), Occ(E)(3)) are true at w 0. Consequently Cause(Occ(C)(1), Occ(E)(3)) is true at w 0 (by Axiom (9)). The most common examples of asymmetrical redundancy are cases of what Lewis calls late preemption. In such cases the preempting cause produces the effect before the preempted alternative can do so. Wright [20] gives the example of two forest fires, A and B approaching a house from opposite directions. Suppose that A arrives first and destroys the house. Then it seems reasonable to conclude that A caused the destruction; despite the fact that if A not done so, then B would have. Halpern and Pearl [9, p. 1] motivate their theory of causation as contingent dependency with this example. They suggest that the house burning down depends on (is caused by) fire A under the contingency that the fire fighters arrive and extinguish fire B before it reaches the house, but not otherwise. This seems wrong. For example, if A was started by arsonist a, B was started by rival arsonist b, and B was not extinguished by the fire fighters, then a would be considered to be guilty of arson, and b of attempted arson. The usual, and it seems correct, solution to cases of this kind is to appeal to the time at which events occur. This is done in the formal treatment of the example in [1, pp. 9-10], A destroys the house at

5 time t thereby preempting B from doing so at the later time t as the house is no longer flammable at t. A preempts B by removing one of B s preconditions, and so the conflict is resolved at the event level (by the event theory). However in cases of trumping preemption this resource is not available, as the two candidate causes are followed by the very same effect. Lewis [12] gives the following example, which was suggested by van Fraassen. Suppose that a sergeant and a major simultaneously order their soldiers to advance and that the soldiers do so. Their advance is redundantly caused, since either order would, on its own, have been sufficient. However, the redundancy is asymmetric, since the soldiers obey the senior officer. The soldiers advance because the major orders them to, not because the sergeant does. The major s order trumps the sergeant s. Once again, the formal treatment [pp ] shows that an event-level resolution is possible by making it a precondition of issuing an order that no more senior soldier issues an order at the same time; with the consequence that the major s order succeeds and the sergeant s fails. An alternative, more flexible, solution is to use an event preference to represent the fact that the senior officer s orders normally override conflicting orders from the junior officer. Redundancy causes difficulty for dependence considered as a sufficient condition. Indeed, the problem of trumping prompted Lewis to abandon simple dependence in favour of the more complex and vague notion of counterfactual covariance [12]. However, it does not appear to pose problems for the Production Theory because conflicts can be resolved at the event level (by the underlying event theory). This seems to be the appropriate place because the resolution of conflicts depends on specific causal knowledge of the kind encoded in the definitions of events and event preferences. This also has the advantage of making it possible to give a simple, general, definition of causation. 6 CONCLUDING REMARKS I agree with Hall that causation as production should be seen as the central metaphysical notion of causation, and suggest that this notion can be formalized by combining appropriate forms of regularity and dependence. The resulting Production Theory captures two significant features of production: regular succession (expected/normally dependable outcome) and making a difference in context (efficacy), and inherits the strengths of regularity and dependence theories while avoiding their weaknesses. The Production Theory also offers a clear division between a general, and relatively simple, definition of causation, and particular causal knowledge embodied in the definition of events and event preferences. As the common sense concept of causation is complex and vague, any comprehensible formal theory of it must inevitably be prescriptive rather than descriptive. In a related paper [2] I consider the more exotic causal notions of prevention and causation by absence, and show that prevention can be reduced to counterfactual production and that causation by absence can be reduced to counterfactual production together with a pragmatic parameter. For example, the ordinary language claim Lack of food causes hunger can analyzed as the counterfactual If food were present, then eating would cause (produce) an absence of hunger provided that it is assumed that eating normally occurs when food and hunger are present. These considerations suggest that the Production Theory is of some philosophical interest. Moreover as the theory is entirely formal, it can be implemented (in simple cases at least) and used by artificial agents to reason about actual and counterfactual causation. ACKNOWLEDGEMENTS I am grateful to the reviewers and to everyone else who has commented on this work. REFERENCES [1] John Bell, Causation and causal conditionals, in Proceedings of the 9th International Conference on the Principles of Knowledge Representation and Reasoning (KR 2004), pp AAAI Press, (2004). [2] John Bell. Causation and absence, Available at: ac.uk/ jb. [3] John Bell. A common sense theory of causation, Available at: www. dcs.qmul.ac.uk/ jb. [4] John Bell. Natural events, Available at: jb. [5] John Collins, Ned Hall, and L. A. Paul (Editors), Causation and Counterfactuals, MIT Press, Cambridge, Massachusetts, [6] R. Fikes and Nils J. Nilsson, STRIPS: A new approach to the application of theorem proving to problem solving, Artificial Intelligence, 2, , (1971). [7] Ned Hall, Causation and the price of transitivity, In Collins et al. [5], pp [8] Ned Hall, Two concepts of causation, In Collins et al. [5], pp [9] Joseph Halpern and Judea Pearl, Causes and explanations: A structural model approach. Part I: Causes, in Proceedings of the Seventeenth Conference on Uncertainty in AI (UAI 2001), pp , (2001). Extended version available at: [10] David Hume, A Treatise of Human Nature, Clarendon Press, Oxford, First published in [11] David Lewis, Philosophical Papers, volume II, Oxford University Press, Oxford, [12] David Lewis, Causation as influence, In Collins et al. [5], pp [13] David Lewis, Void and object, In Collins et al. [5], pp [14] John McCarthy, Applications of circumscription to formalizing commonsense knowledge, Artificial Intelligence, 28, , (1986). [15] J.S. Mill, A System of Logic, Longmans, London, First published in [16] Raymond Reiter, Knowledge in Action; Logical Foundations for Specifying and Implementing Dynamical Systems, MIT Press, Cambridge, Mass., [17] Murray Shanahan, Solving the Frame Problem; A Mathematical Investigation of the Common Sense Law of Inertia, MIT Press, Cambridge, Mass., [18] Yoav Shoham, Reasoning About Change, MIT Press, Cambridge, Mass., [19] Yoav Shoham, Nonmonotonic reasoning and causation, Cognitive Science, 14, , (1990). [20] R. Wright, Causation, responsibility, risk, probability, naked statistics and proof: Pruning the bramble bush by clarifying the concepts, Iowa Law Review, 73, , (1988).