Social Science Counterfactuals. Julian Reiss, Durham University

Social Science Counterfactuals Julian Reiss, Durham University

Counterfactuals in Social Science Stand-ins for causal claims about singular events Stand-ins for causal claims in the potentialoutcomes approach in statistics What-if fictions Counterfactual definitions in economics (e.g., the COLI index) What-if counterfactuals for policy

Starting Points Counterfactuals come in different varieties, and they are ubiquitous in the social sciences It would be as futile as it would be harmful to try to get rid of them One important purpose for which they are employed is causal inference: a true counterfactual claim can be evidence for a causal hypothesis That this is so, follows naturally from a counterfactual theory of causation

Starting Points Counterfactual dependence is neither necessary nor sufficient for causation But the two are inferentially related: under some conditions, we can reliably infer a claim about singular causation from a claim about counterfactual dependence, and (under perhaps other conditions) vice versa So here I ask: If we were to use claims about counterfactual dependence as evidence for causal claims in the social sciences, how would we best go about knowing their truth, and what challenges would we face?

Max Weber s Theory of Causal Inference From Objective Possibility and Adequate Causation in Historical Explanation : Rather, does the attribution of effects to causes take place through a process of thought which includes a series of abstractions. The first and decisive one occurs when we conceive of one or a few of the actual causal components as modified in a certain direction and then ask ourselves whether under the conditions which have been thus changed, the same effect... or some other effect would be expected. Suppose Weber were right how would we know if under the conditions which have been thus changed, the same... or some other effect would be expected?

Approaches to evaluating counterfactuals Among philosophers and social scientists three approaches have dominated thinking about counterfactuals: Possible-worlds approaches (Stalnaker, Lewis) Cotenability approaches (Goodman, social scientists) Causal-model approaches (Dretske, Pearl, Reiss and Cartwright, Hiddleston) Can we use any of these for inference?

Lewis... aimed to substitute the regularity view of causation with a neo-humean alternative without piling on the epicycles Ironically, over 35 years on and after more than one pile of epicycles, Lewisstyle theories cannot even come to terms with those problems that motivated the counterfactual theory in the first place: effects, epiphenomena, pre-emption Ignoring cases of redundant causation, Lewis theory says that if one event E counterfactually depends on an independent event C, C causes E And: E counterfactually depends on C iff there is no C, E-world that is more similar to ours than any C, E-world Enormous confidence: we can (weakly) order possible worlds wrt similarity just as we can order faces, cities, philosophies

Lewis Two main problems: Suppose two historians disagree, how could we resolve differences? Doesn t seem to get the causal claims right: Kit Fine s Nixon Thus, Lewis develops a more detailed account of overall similarity ((1) avoid big miracles, (2) maximise exact match of facts, (3) avoid small miracles, (4) don t worry about approximate match of facts) Two problems remain: There are few if any known laws Still problems with the semantics: Morgenbesser s coin

... specify six tests: Tetlock and Belkin 1. Clear antecedents and consequents 2. Contenability 3. Minimal rewrite rule 4. Theoretical consistency 5. Statistical consistency 6. Projectability These are intuitively very plausible guidelines; but they too are vague and not always easy to pass

Tetlock and Belkin e.g., the minimal rewrite rule: possible worlds should (a) start with the real world as it was otherwise known...; (b) not require us to unwind the past and rewrite long stretches of history; (c) not unduly disturb what we otherwise know about the original actors and their beliefs and goals Vagueness is evident: long, unduly Tradeoff: the further we go back into the past, the more likely it is that we find an event in the causal history of the antecedent that could have been otherwise without violating what we know about this history Theoretical and statistical consistency are too weak and too strong at the same time

Causal Theories of Counterfactuals... are intuitively appealing: counterfactual claims seem to be grounded in causal knowledge... are epistemically appealing: there are wellestablished methods of causal inference Hiddleston 2005 provides a semantics that not only gets the philosophers toy cases right, it is also more consistent (than Lewis) with historical practice

A Causal Theory of Counterfactuals Counterfactuals are defined within causal models: M = <G, E, A>, with equations assigning probabilities to variables given their direct causes X = x has direct causal influence on Y = y in M iff p(y = y X = x, Z = z) > p(y = y X x, Z = z) A causal break M Mi is a variable Y such that Ai(Y) A(Y) and for each parent, Ai(X) = A(X) Break(M i, M) = {Y: Y is a causal break in Mi from M} Intact(M i, M) = {Y: Ai(Y) = A(Y) and for each positive parent, Ai(X) = A(X)}

A Causal Theory of Counterfactuals PM Counterfactuals are defined within causal models: BFP M = <G, E, A>, with Cab equations assigning probabilities to variables given their direct causes Ai(PM) = {Chamberlain, Cooper, Eden, Churchill} Ai(BFP) = {Dove, Hawk} X = x has direct causal influence on Y = y in M iff p(y = y X = x, Z = z) > p(y = y X x, Z = z) A causal break M Mi Ai(Cab) is a variable = {Doves, Y Hawks} such that Ai(Y) A(Y) and for each parent, Ai(X) = A(X) BFP = Dove iff PM = Chamberlain Break(M i, M) = or {Y: Cab Y is = a Doves causal break in Mi from M} Intact(M... i, M) = {Y: Ai(Y) = A(Y) and for each positive parent, Ai(X) = A(X)}

A Causal Theory of Counterfactuals ϕ-minimal Model: Model Mi and Break(Mi, M) are ϕ-minimal relative to M iff (i) Mi is a ϕ-model (ii) for Z, the set of non-descendants of ϕ, Intact(Mi, M) Z is maximal among ϕ-models (iii) Break(Mi, M) is minimal among ϕ-models TCM: (ϕ > ψ) is true in a model iff ψ is true in every ϕ- minimal model Mi

A Causal Theory of Counterfactuals ϕ-minimal Model: Model Mi and Break(Mi, M) are ϕ-minimal relative to M iff (i) Mi is a ϕ-model Bet = T (ii) for Z, the set of non-descendants of ϕ, Intact(Mi, M) Z is maximal among ϕ-models Toss = 1 (iii) Break(Mi, M) is minimal among ϕ-models TCM: (ϕ > ψ) is true in a model iff ψ is true in every ϕ- minimal model Mi Win = 0 Outcome = H

A Causal Theory of Counterfactuals ϕ-minimal Model: Model Mi and Break(Mi, M) are ϕ-minimal relative to M iff (i) Mi is a ϕ-model (ii) for Z, the set of non-descendants of ϕ, Intact(Mi, M) Z is maximal among ϕ-models (iii) Break(Mi, M) is minimal among ϕ-models TCM: Intact (ϕ > ψ) is true in a model iff ψ is true in every ϕ- minimal model Mi Bet = TH Toss = 1 Break Win = 10 Outcome = H

TCM vs Philosophers TCM differs importantly from Lewis and others in the philosophical tradition Unlike Lewis, Pearl, Hitchcock and others (but like some historians/social scientists), TCM implements the counterfactual antecedent not by miracle (exchanging one structural eqation for another) but by changing the value for a causal variable in an existing equation Since the counterfactual value of the causal variable must be possible given the equations and the values of the variable s parents (p(a(x) A(pa(X))>0), this often means that counterfactuals backtrack Interestingly, this is just what we observe in historical practice So the account is descriptively sound. But is it epistemically and semantically sound?

TCM vs Philosophers TCM differs importantly from Lewis and others in the philosophical tradition Unlike Lewis, Pearl, Hitchcock and others (but like some historians/social Flash scientists), TCM implements the counterfactual antecedent not by miracle (exchanging one structural Fuse lit eqation Explosion for another) but by changing the value for a causal variable in an existing equation Bang Since the counterfactual value of the causal variable must be possible given the equations and (a) the (Fuse values lit = of 1) the p(explosion variable s = parents 1) =.95 (p(a(x) A(pa(X))>0), this often means that counterfactuals backtrack (b) Flash = Explosion (c) Bang = Explosion Interestingly, this is just what we observe in historical practice So the account is descriptively sound. But is it epistemically and semantically sound?

TCM vs Philosophers TCM differs importantly from Lewis and others in the philosophical tradition Unlike Lewis, Pearl, Hitchcock and others (but like some historians/social scientists), TCM implements PM the BFP counterfactual War antecedent not by miracle (exchanging one structural eqation for another) but by changing the value for a causal variable in an existing equation Since the counterfactual P(BFP = value Hawk PM of the = Chamberlain) causal variable = 0must be possible given the equations and the Φ = values Had British of the foreign variable s policy been parents (p(a(x) A(pa(X))>0), this often means that different counterfactuals backtrack Break(Mi, M) = PM [rather than BFP] Interestingly, this is just what we observe in historical practice So the account is descriptively sound. But is it epistemically and semantically sound?

Some problems for the account There are at least five potential sources of error: The problem of circularity The problem of backtracking The problem of actual cause The problem of indeterminacy The problem of confirmation

Problem I: Circularity The account is obviously circular does that affect its usefulness? Recent work on causation shows that accounts can be circular and yet useful (Cartwright, Woodward): No causes in, no causes out There are two reasons why, while circular, the account nevertheless might help with inference Semantic: the causal laws require an assignment of probabilities to values of variables given the values of those variables that causally influence it but whether or not something is an actual cause is a model implication rather than built into it Epistemic: when causal relations become complex, these such implications might not be obvious

Problem I: Circularity The account is obviously circular does that affect its usefulness? Recent work on causation shows that accounts can be circular and yet useful (Cartwright, Woodward): No causes in, no causes out There are two reasons why, while circular, the account nevertheless Cab might help with inference BFP War Semantic: the PMcausal laws require an assignment of probabilities to values of variables given the values of those variables that causally influence it but whether or not something is an actual cause is a model implication rather than built into it Epistemic: when causal relations become complex, these such implications might not be obvious

Problem I: Circularity The account is obviously circular does that affect its usefulness? Recent work on causation shows that accounts can be circular and yet useful (Cartwright, Woodward): No causes in, no CDE causes out There are two reasons why, while circular, War38 the account nevertheless might help with inference Cab BFP Det Semantic: the causal laws require an assignment of probabilities to PM values of variables given the values of those variables that causally influence it but whether or not something is an actual War39-45 cause is a model implication rather than built into it Epistemic: when causal relations become complex, these such implications might not be obvious

Problem IV: Indeterminacy A very serious problem with the account (for causal inference) is that by and large the counterfactual will be of the might have been kind: If Britain had been more resolved over the Sudetenland, World War II might have been avoided This is true once any of the structural equations connecting antecedent and consequent are indeterministic Which is the case more often than not E.g., in the model shown above, with plausible estimates of the probabilities, BFP = hawk p(war39-45) =.12 (from.7) Might have been counterfactuals are not evidence for causal claims Possible solution: probabilistic theory

Problem V: Confirmation Because of (IV), we want deterministic models (except for antecedent) But even in the deterministic case, to show that her model is a good one, the historian/social scientist has to provide evidence that A. C, E exist B. C has the causal power to bring about E C. No other factors C that have the causal power to bring about E were operative... because they are absent... because their operation was interrupted before E... because C pre-empted them

Problem V: Confirmation The main problem lies with (B) This is not because of general scepticism concerning evidence for causal powers ( causings ) But effects of interest in history and other social sciences are usually macro events Macro causings are (almost?) never observable But then how could we know that C s can, on occasion, bring about E s? Process tracing? Stays at micro level Statistics, QCA? Reference class, small-n problems

The virtues of counterfactual speculation The real benefit reaped from evaluating counterfactuals using TCM is what is learned in the process of model construction and attempted confirmation What does the antecedent assert? (Construct a variable.) Can the antecedent be implemented given what we know about the world, and how? How are antecedent and consequent connected? What are the mediating variables? Are the relations deterministic or not? Is the consequent dependent on different specifications of the antecedent and different ways of bringing it about? These model-building exercises might also help with numerous cognitive biases Alternatives such as process tracing are different but not necessarily better

Julian Reiss Durham University julian.reiss@durham.ac.uk www.jreiss.org