Coherence and probability in legal evidence

Abstract

The authors investigate to what extent an evaluation of legal evidence in terms of coherence (suggested by Thagard, Amaya, Van Koppen and others) is reconcilable with a probabilistic (Bayesian) approach to legal evidence. The article is written by one author (Dahlman) with a background in the bayesian approach to legal evidence, and one author (Mackor) with a background in scenario theory. The authors find common ground but partly diverge in their conclusions. Their findings give support to the claim (reductionism) that coherence can be translated into probability without loss. Dahlman therefore concludes that the probabilistic vocabulary is superior to the coherence vocabulary, since it is more precise. Mackor is more agnostic in her conclusions about reductionism. In Mackor's view, the findings of their joint investigation do not imply that the probabilistic approach is superior to the coherentist approach.

1. Introduction

Some scholars claim that legal fact-finders should evaluate evidence in terms of coherence (MacCormick 1980; Wagenaar, van Koppen Crombag 1993; Thagard 2000, 2004; Bex 2011; Amaya 2013). Within the coherence approach various theories can be distinguished, among others the scenario theory advocated by Peter van Koppen (2011, also see Van Koppen and Mackor 2019) and the theory of explanatory coherence advocated by Paul Thagard (2000) and Amalia Amaya (2013, 2015) who combine it with inference to the best explanation (IBE) and propose that IBE should be imported into law and adapted to legal fact-finding.

Both IBE and the coherence approach have been criticized by scholars who say that legal fact-finders should evaluate evidence in probabilistic (Bayesian) terms. Bayesian scholars often say that IBE and the coherence approach only provide a fuzzy vocabulary for saying things that are captured in a more precise way by the Bayesian approach. In this article, we will refer to the view that coherence can be reduced to probability as reductionism. According to reductionism, coherence can be defined in probabilistic terms without loss. Reductionism has been met by strong objections from advocates of the coherence approach, claiming that coherence cannot be reduced to probability. We will refer to this view as non-reductionism. According to non-reductionism, something essential is lost when coherence is reduced to probability.

This article differs from other papers that offer an analysis of the relation between coherence and probability. Most papers analyse the relation between the concepts of coherence and probability, without taking into account the theories about reasoning about evidence in law in which coherence plays a central role. This article first explicates the concept of coherence as it is actually used in two important theories about reasoning about evidence in law and then investigates whether this specific concept can be explicated in probabilistic terms. Moreover, in this article, we explicitly distinguish between the concept of coherence and coherentist theories. We claim that even if coherence is reducible to probability, it does not follow that coherentist theories are reducible to or redundant in relation to probabilistic theories.

The structure of this article is as follows. In Section 2, we start with an exposition of two theories in which coherence plays a pivotal role, viz Van Koppen’s scenario theory and Amaya’s theory of inference to the most coherent explanation which builds on Thagard’s analysis of explanatory coherence (Sections 2.1 and 2.2) and we propose some improvements (Section 2.3).

Next, we explore the idea that the coherence vocabulary, more specifically the vocabulary that has been expounded in Section 2, can be reconciled with the Bayesian vocabulary by defining coherence in probabilistic terms as positive relevance (Section 3). Subsequently, we discuss an argument that has been presented by Mark Siebel for non-reductionism, viz. that probability cannot fulfill the role that coherence plays in explanation (Section 4). We analyse the role of inference to the best explanation (IBE) and stress the importance of distinguishing the relation between the concepts of coherence and probability from the relation between a coherentist and a probabilistic approach (Section 5). We end with some concluding remarks (Section 6).

The article is a collaboration between one scholar with a background in the scenario approach (Mackor) and one scholar with a background in the Bayesian approach (Dahlman). It should be read as an exploration not only of the possibility to reduce coherence to probability but also to reconcile these approaches. As mentioned above, there is a debate between Bayesians and coherentists over the possibility to define coherence in probabilistic terms (reductionism versus non-reductionism). We explore this possibility, and show how coherence can be reduced to probability (reductionism). However, we also argue that this does not imply that the coherentist approach should be reduced to a probabilistic approach.

2. The coherence approach

In this section, we present two theories about the justification of evidentiary judgments in criminal law in which coherence plays a central role (Sections 2.1 and 2.2) and offer a few suggestions about how they can be combined and improved (Section 2.3). The first theory we expound is the scenario theory of Van Koppen (2011, 2019). The second is the coherence theory of Amaya (2013, 2015) that builds upon Thagard’s analysis of explanatory coherence (2000). We have chosen to discuss only these two theories, because these theories, in particular Amaya’s theory that builds on Thagard’s concept of coherence, offer the most elaborate analyses both of the concept of coherence and of the role coherence plays in theories about reasoning about evidence in law.¹

2.1 The scenario theory of Van Koppen

Van Koppen’s scenario theory (2011, 2019) is an elaboration of an earlier theory of Wagenaar et al. (1993). The theory is based on the story model of Pennington and Hastie (1993). The story model is a descriptive psychological theory that Wagenaar et al. (1993) have given a normative twist. We will first expound the story model before we turn to Van Koppen’s theory.

2.1.1 The story model of Pennington and Hastie

The story model is a descriptive psychological theory about the cognitive strategies that fact-finders use to process trial information in order to decide. One central strategy is active story construction: fact-finders impose a narrative story organization on the trial information (Pennington and Hastie 1993, 194).

The story model offers both an analysis of the structure of stories and of the way in which people construct and reason about stories.

2.1.2 The structure of stories

Stories consist of elements, which are called episodes. Episodes consist of specific elements, viz. an initiating event, a psychological response, (sometimes a goal), an action and a consequence. Episodes have a specific structure: the elements are chronologically connected through physical and mental causal relationships. Stories can be thought of as a hierarchy of embedded episodes (Pennington and Hastie 1993, 197). An example of an episode that is a simple story: a husband has an argument with his wife (initiating event), which makes him angry (psychological response). Out of anger (psychological response) (or also because he intends to hurt or kill her (goal), he beats his wife (action)), which causes her death (consequence).

2.1.3 The construction of stories²

Fact-finders construct stories by reasoning from three kinds of knowledge. They use:

• case-specific knowledge, i.e. evidence;

• knowledge about similar events to infer facts and causal relationships;

• knowledge about what makes a story complete (viz. knowledge about the elements of stories, viz. episodes and their elements, and about the connections in and between episodes) (Pennington and Hastie 1993, 194).

Fact-finders use (b) and (c) to ‘fil out’ the story. Thus, a story consists of (facts corresponding to³) evidence, inferred facts and causal relations between them.

2.1.4 Reasoning procedures

Pennington and Hastie (1993, 195) mention in particular three types of reasoning procedures that fact-finders use to establish (intermediate) conclusions:

• deductive reasoning from world knowledge;

• reasoning from analogy to other—experienced and hypothetical—episodes;

• reasoning by evaluating alternate conclusions that contradict the initial conclusion.

2.1.5 The assessment of stories⁴

Fact-finders use three certainty principles to assess stories, viz. coverage, coherence and uniqueness. These principles help a fact trier to determine how acceptable a story is for him and how confident he is about the truth of the story (Pennington and Hastie 1993, 198–9).

1. Coverage deals with the question to what extent the story accounts for, i.e. explains, the evidence. The greater the coverage, the more acceptable the story and the more confident the fact-finder will be.

2. A story is unique if it is the only coherent story that can account for the evidence. If there is more than one coherent story, all stories are in principle acceptable, but confidence in them will diminish.

3. Coherence has three components: consistency, plausibility and completeness.

Consistency is about two questions, viz.

1. whether the story is consistent with (specific) evidence believed to be true and

2. whether it is consistent with other parts of the explanation, i.e. of the story.

Plausibility deals with the question whether the story is consistent⁵ with the fact-finders’ background or world knowledge.

Completeness, finally, is about the question whether the structure of the story has all its parts, i.e. episodes, elements of episodes and causal relationships in and between episodes. Missing information and/or lack of plausible inferences makes a story incomplete and decreases confidence in the story.

Pennington and Hastie state that consistency, plausibility and completeness can be fulfilled to a greater or lesser degree and that the values of the three components combine to yield the overall coherence of the story (Pennington and Hastie 1993, 199). However, they do not give further specification of the components of coherence, nor of the way their individual weight is assessed and the way in which they are combined. Moreover, it is questionable whether their analysis of coherence, which is only in terms of consistency and completeness, is adequate given that a consistent set of statements need not be coherent. We say more about this in Section 2.2.

2.1.6 Van Koppen’s scenario theory

Wagenaar et al. (1993) have converted the descriptive psychological theory of Pennington and Hastie about how people actually reason into a normative theory about how people should reason and make decisions about evidentiary judgments in law if they want to do so in a rational manner.

The obvious advantage of a normative theory that builds on a descriptive theory is that it will be more easily accepted and used in actual practice than theories, such as Bayesian probabilistic theories, that are not closely related to, or even conflict with, how people actually reason.⁶

If one turns the descriptive theory into a normative theory, a first step could be to explicitly instruct fact-finders to construct and assess the story in accordance with Pennington and Hastie’s theory and to use their ‘certainty principles’ as normative guidelines. However, an obvious critique of the shift from a descriptive to a normative theory is that it presumes that people in fact use the proper concepts and methods to assess and decide about evidence in criminal trials. Crombag, Van Koppen and Wagenaar, all three being psychologists, were fully aware of the fact that people can fall prey to all sorts of biases. Van Koppen (2011, 60) for example emphasizes that people suffer from confirmation bias and belief perseverance and that they avoid cognitive dissonance.

2.1.7 Guidelines and critical questions

Accordingly, what a normative story model should do is to offer guidelines and critical questions that help to prevent people from falling prey to biases. Van Koppen (2011, 64) states that when fact-finders create or assess a scenario, they should choose a Popperian stance.

1. They should not (only) seek evidential confirmation for the scenario under investigation, but also try to falsify it (i.e. seek contradiction).

2. They should do so by creating not just one scenario, but also create alternative scenarios and assess how well they explain the evidence. In particular, they should look for discriminating facts, i.e. evidence that can be explained by (is coherent with) one scenario, but not (contradiction) or less well (incoherence) by other scenarios.

Thus, coherence, and because of Van Koppen’s Popperian stance, contradiction and incoherence play a vital role in the normative version of the story model. Van Koppen (2011), however, does not offer a more refined analysis of the concept of coherence than Pennington and Hastie (1993).

2.2 Amaya’s theory of inference to the most coherent explanation

Possibly the most detailed analysis of coherence and the role it should play in inferences in evidential judgments in law is Amaya’s coherence theory (2013, 2015), which builds on Thagard’s theory of explanatory coherence (2000). The theory offers an analysis both of the concept of coherence and of the process of coherence maximization. Coherence is claimed to play a role in the generation and the pursuit as well as in the justification of explanations.

The two most important ingredients of Amaya’s theory are coherence and IBE.⁷ On her view, the best explanation is the one that does best on a test of coherence (2009, 137). We use the abbreviation ICE (Inference to the most Coherent Explanation) to refer to her theory.

2.2.1 Coherence

Amaya relies on Thagard’s analysis of coherence, in particular on his analysis of explanatory coherence. Thagard (2000, 15 ff) defines coherence as the satisfaction of a set of positive and negative constraints (coherence and incoherence relations) among a set of elements. The set of elements E is divided into two disjoint subsets A (accepted) and R (rejected).

An obvious problem with the idea of constraint-satisfaction is that the constraints are not equally important and must therefore be weighted. According to what criteria should these weights be assigned? As we shall see, Thagard’s theory does not answer this question.

The main elements in the assessment of evidentiary judgments in a criminal trial are hypotheses (H) and evidence (e). The main, but not the only, type of coherence involved in these judgments is explanatory coherence.⁸

Thagard (2000, 43) distinguishes seven principles of explanatory coherence. Amaya states that when applying Thagard’s theory to a particular problem, it must be further specified and contextualized. Therefore, in her application of Thagard’s theory in the context of criminal law, she adds two coherence principles, viz. to E4 and to E7 (see below) and she stresses that next to explanatory coherence, deliberative coherence should play a role too. She does not, however, offer a further specification of the elements (H and E) in the set.⁹

2.2.2 Seven principles of explanatory coherence

We will now state Thagard’s seven principles of explanatory coherence and the two sub-principles Amaya adds to them.

E1 symmetry

Explanatory coherence is a symmetrical relation, unlike explanatory relations and relations of conditional probability.¹⁰

E2 explanation

• an hypothesis (H) coheres with what it explains: evidence (E) or another hypothesis (H);

• hypotheses that together explain another proposition (H or E) cohere;

• the more hypotheses needed to explain something, the lower the degree of coherence.¹¹

E3 analogy

Similar hypotheses that explain similar pieces of evidence cohere.¹²

E4 data priority

Propositions that describe the results of observations (evidence) have a degree of acceptability on their own.

For the context of criminal law, Amaya (2013, 13) adds the principle that factual hypotheses that are compatible with innocence have a degree of acceptability on their own.¹³

E5 contradiction

Contradictory propositions are incoherent with each other.¹⁴

E6 competition

Hypotheses that both explain a proposition but that are not explanatorily connected (cf. principle E2b) are incoherent (i.e. also if they do not contradict each other).

E7 acceptance

The acceptability of a proposition in a system depends on its coherence with other propositions.

For the context of criminal law Amaya (2013, 13) adds the principle that the guilt hypothesis may be accepted only if it is justified to a degree sufficient to satisfy the reasonable doubt standard.¹⁵

2.2.3 Inference to the most coherent explanation

Amaya (2009, 2013) also offers an analysis of the process of coherence maximization. She argues that this process is an explanatory inference, which fits the model of IBE.¹⁶ Amaya uses Lycan’s (1988 and 2002) definition of IBE:

Fq … Fn are facts in need of explanation

Hypothesis H explains F1 … Fn

No available competing hypothesis explains Fi as well as H does

Therefore, probably H is true

As was stated above, Amaya’s criterion of best is best on a coherence test. She states that an inference to the most coherent explanation consists of the following explanatory inference steps (2013, 16)¹⁷:

1. the specification of a base of coherence, i.e. the set of factual hypotheses and evidence over which the coherence calculus proceeds;

2. the construction of a contrast set that contains a number of alternative theories from which the most coherent is to be selected;

3. refining and revising the alternative theories by means of coherence-making mechanisms, in particular addition, subtraction and reinterpretation.¹⁸ This can result in a revision of the contrast set (2), but it can also lead to revision of the base set (1);

4. the evaluation of the coherence of the alternative theories by means of the principles of explanatory coherence E1–E7;

5. the selection as justified of the most coherent theory, provided that its degree of justification satisfies the applicable legal standard of proof.

2.2.4 Coherence versus intuition

Amaya explicitly considers the possibility that the theory that best satisfies the criteria of coherence nevertheless seems intuitively unjustified (2013, 18).¹⁹ She points out that errors might have been made in step 1, the selection of the elements, i.e. of the base set of hypotheses and evidence. In particular, it is possible that relevant evidence has been ignored. In step 2, the constructing of the contrast set, fact finders can fail too, as a consequence of which they will at best end up with an inference to the best of a bad lot (Van Fraassen, 1989).

Problems can also arise in the inference to the most coherent explanation (step 4). Amaya points out that people suffer from coherence bias, that is, they inflate some alternatives and deflate others in order to maximize coherence.²⁰ By ignoring or misrepresenting evidence or alternative hypotheses they distort the set of evidence and hypotheses that threaten their beliefs.

How can fact-finders prevent these errors? In essence they do so by choosing an eliminative approach.²¹ We have seen that Van Koppen (2011) instructs fact finders to actively construct alternative scenarios and to actively seek for the falsification of all scenarios. Likewise, Amaya mentions the duty to actively search for alternative hypotheses (2013, 26), but she does not stress the importance of attempts at falsification of theories.²²

2.3 ICS: refining the scenario theory and the theory of ICE

In this section we bring together our findings about the two theories that we have just discussed, the scenario theory and the theory of ICE and investigate whether they can be fruitfully combined into an ‘inference to the most coherent scenario’ theory (ICS). Since Amaya offers a more refined analysis of coherence we take Amaya’s theory of ICE as a starting point and investigate whether elements of the scenario theory can be integrated in her theory. We believe that a combined version of these theories offers the most elaborate exposition of coherence and its role in the assessment of evidence in legal cases. Therefore, this improved version is the starting point of Section 3 in which we investigate whether coherence can be explicated in probabilistic terms.

2.3.1 Hypotheses, episodes and completeness

The most important question seems to be whether the elements of stories hat Pennington and Hastie distinguish, viz. episodes and their elements and the chronological and causal relationships between them, can and should be integrated in the theory of ICE. Not all legal cases are or need to be structured as stories, so integration of the elements of stories into the theory of ICE is not always necessary. In our view, however, they should be integrated in criminal cases, since these are normally structured as stories. Here we investigate how such integration can take place. More in particular, we investigate whether these elements can be used to refine Thagard’s explanatory principle E2, in particular E2 sub (b) (the principle that hypotheses that together explain another proposition cohere) and Amaya’s analysis of the contrast set that contains a number of alternative theories from which the most coherent is to be selected in explanatory inference step 2).

On the one hand, we have seen that Thagard’s (and Amaya’s) analysis of coherence is much more refined than the analysis of Pennington and Hastie who largely equate coherence to 1) the internal consistency of the story, 2) the consistency of the story with the evidence and 3) the consistency with general knowledge. Thagard (and Amaya) define coherence in terms of constraints and offer a set of principles of explanatory coherence, which show that coherence is much more than consistency.

On the other hand, the scenario theory, which incorporates the story model of Pennington and Hastie, is more refined than Amaya’s theory when it comes to the analysis of the different elements of a story. Thagard and Amaya do not talk about stories but about theories and they only distinguish between hypotheses and evidence. Pennington and Hastie do not use the term ‘hypothesis’. However, descriptions of episodes and their elements are specific kinds of hypotheses, viz. about 1) an initiating event, 2) a psychological response, (3) a goal), 4) an action and 5) a consequence. Moreover, these elements stand in a specific, viz. in a chronological and a (mental or physical) causal relationship to each other. If we want to take these insights on board, Pennington and Hastie’s certainty principle of completeness, which is a check on missing information about the elements of the episodes and lack of plausible inferences, should be added to Thagard’s set of explanatory coherence principles and Amaya’s set of inference steps.

Accordingly, we propose that Amaya’s explanatory inference step 2 should be refined. Inference step 2 would then become: construct a contrast set that contains a number of alternative scenarios that have all of their elements which are properly connected. The same holds for the hypotheses mentioned in Thagard’s explanatory coherence principle E2. The story model instructs us to distinguish (at least) five kinds of hypotheses about elements that should be properly, viz. chronologically and causally, connected. Principle E2 should thus instruct us to check whether all hypotheses are formulated and whether they are properly, i.e. chronologically and causally, connected. Accordingly, Pennington’s and Hastie’s principle of completeness²³ should be explicitly added to Thagard’s set of principles and it should be incorporated in Amaya’s step 2 of the construction of the contrast set.

There is another refinement of the combination of the scenario theory and the ICE theory that we want to discuss. We elaborate on Thagard’s explanatory principle E2 sub c), which expresses the epistemic virtue of simplicity, by exploring the possibility of assessing scenarios in terms of epistemic virtues.

2.3.2 Epistemic virtues

Thagard’s principle of explanatory coherence E2 sub c) states ‘the more hypotheses it takes to explain something the lower the degree of coherence’. This is an explication of the epistemic virtue of simplicity in terms of coherence. If it is possible to analyse other epistemic virtues such as empirical adequacy, simplicity, unification, precision,²⁴ stability and robustness in terms of coherence, we can incorporate epistemic virtues into the theory of ICS.

Why do we want to add the notion of epistemic virtue to a coherence approach? In epistemology and philosophy of science, more specifically in theories about IBE, epistemic virtues are seen as properties of theories that scientists do and should use to assess the quality of scientific theories and to make a rational choice between them. Several attempts have been made to explicate virtues like simplicity in terms of coherence (Thagard 1978; McMullin 1996; Lipton 2001, 2004; Douglas 2013; Mackonis 2013; Cabrera 2017; Keas 2017).

2.3.3 Descriptive and normative theories about epistemic virtues

There is an interesting parallel between the story model and theories about epistemic virtues. Some philosophers defend the descriptive psychological claim that people intuitively assess the quality of explanations in terms of epistemic virtues. Moreover, it is argued that epistemic virtues seem cognitively more easily accessible than priors and likelihoods (Lipton 2001, 111). Thus, a normative theory that explicates these virtues in terms of coherence and creates and refines normative guidelines and critical questions that help people to assess explanations in terms of epistemic virtues, might have the advantage that it is easier to use in daily practice than a Bayesian approach.²⁵

For our article, the interesting question is not only whether epistemic virtues can offer guidance to fact finders to assess scenarios and whether virtues can be explicated in terms of coherence, but also whether they can in turn be explicated in probabilistic terms as for instance T. McGrew (2003) and Cabrera (2017) argue.

Lipton (2001, 2004) has made the well-known distinction between loveliness (understanding) and likeliness (posterior probability). He argues that the best explanation is not the most likely explanation, but the loveliest explanation, i.e. the explanation that—if true—gives us most understanding. He also hypothesizes that loveliness is a guide to, but not the same as, likeliness. He furthermore argues that explanatory virtues contribute to our understanding and that explanatory considerations ‘might help to lubricate the Bayesian mechanism’ in guiding the determination of priors, likelihood and relevant evidence.

Douglas (2013), Cabrera (2017) and others, have argued that we should distinguish between confirmational and informational virtues. Confirmational virtues such as explanatory power and robustness contribute both to our understanding of an explanation and to the probability of a hypothesis or a scenario. Thus, in our view, if coherence can be explicated in terms of probability, then it should be possible not only to explicate epistemic virtues in terms of coherence, but also in terms of probability. Informational virtues such as fertility and precision on the other hand do not in themselves contribute to the probability of a hypothesis or a scenario. They play a role in the context of generation and pursuit and help to choose the scenarios that are sufficiently fruitful or fertile to deserve further investigation. Accordingly, it should be possible to analyse informational virtues in terms of coherence, in particular in terms of the coherence of the different hypotheses within a scenario and in terms of the coherence of hypotheses with the evidence that is predicted by the scenario but that has not (yet) been confirmed. Accordingly, although informational values cannot be explicated in terms of actual probability, it should be possible to explicate them in terms of potential probability (i.e. probability if the predicted facts are true).

Now that we have offered an explication of the concept of coherence and the role it plays in two well-known theories about reasoning about evidence in law, we turn to the question whether the concept of coherence as it was expounded in this section can be explicated in probabilistic terms.

3. Reductionism

Reductionism, with regard to coherence and probability, is the view that coherence can be reduced to probability. According to reductionism, everything that is said in terms of coherence in the evaluation of evidence can be said in terms of probability without loss. Advocates of reductionism include Shogenji (1999), Glass (2002), Fitelson (2003) and Olsson (2005).

Reductionism goes back to C.I. Lewis, who proposed that ‘congruence’ between factual beliefs can be defined in probabilistic terms as positive relevance (Lewis 1946, 338). This general idea has been used by reductionists as a basis for a probabilistic account of coherence (Shogenji 1999, 340; Schubert and Olsson 2013, 35; McGrew 2016, 335). A and B are positively relevant for each other if A increases the probability of B, and B increases the probability of A. So, in this view, A and B are coherent with each other if P(B|A) > P(B) and P(A|B) > P(A).

If two beliefs are not positively relevant it is often the case that they are irrelevant for each other, i.e. P(B|A) = P(B) and P(A|B) = P(A), but they could also be negatively relevant, i.e. P(B|A) < P(B) and P(A|B) < P(A). In either case, they are incoherent with each other. Negative relevance corresponds to Thagard′s ‘principle of contradiction’ (E5) discussed above (Section 2.2). This is a notable difference between the coherence vocabulary and the probabilistic vocabulary. A three-valued vocabulary (positive relevance/irrelevance/negative relevance) is more precise than a two-valued vocabulary (coherence/incoherence). This is one of several features that makes the probability vocabulary more precise than the coherence vocabulary.

As we have seen above, coherence theory distinguishes between EH-coherence (coherence between a piece of evidence and a hypothesis) and HH-coherence (coherence between two hypotheses). To this categorization, we can add EE-coherence (coherence between two pieces of evidence). When coherence is understood as positive relevance this translates into a probability vocabulary in the following way.

EH-coherence. A piece of evidence (E) is coherent with a hypothesis (H) if P(H|E) > P(H) and P(E|H) > P(E). This corresponds to Thagard´s ‘principle of symmetry’ (E1) discussed above (Section 2.2). See example of Bayesian network in Fig. 1.

In a criminal trial, the main hypothesis is the prosecutor’s accusation against the defendant, for example, ‘the defendant killed the victim by shooting him in the head’, and the piece of evidence coherent with this hypothesis could for example be a witness that identifies the defendant as the shooter. This corresponds to Thagard´s ‘principle of explanation’ (E2 sub a) discussed above (Section 2.2). The hypothesis ‘explains’ the evidence.

EE-coherence. Two pieces of evidence (E₁ and E₂) are coherent with each other if P(E₁|E₂) > P(E₁) and P(E₂|E₁) > P(E₂). If two pieces of evidence are coherent (positively relevant) with the same hypothesis, it follows from Bayes Theorem that they are coherent (positively relevant) with each other. Since E₁ is positively relevant for H, the instantiation of E₁ increases the probability of H, which, in turn, increases the probability of E₂, since H is positively relevant for E₂. And the same goes in the other direction, from E₂ to E₁. This is known as ‘propagation’ in Bayesian networks. See example in Fig. 2.

In a criminal trial, E₁ could be a witness that identifies the defendant as the shooter, and E₂ could be a forensic report about gunshot residue on the defendant’s hand.

HH-coherence. Two hypotheses (H₁ and H₂) are coherent if P(H₁|H₂) > P(H₁) and P(H₂|H₁) > P(H₂). Figure 3 shows a case where a piece of evidence supports a hypothesis that supports another hypothesis. H₁, H₂ and E are all positively relevant to each other, and are coherent with each other.

This is familiar in criminal evidence, and is often referred to as a ‘chain’. As an example, H₁ could be the hypothesis that the defendant killed the victim, H₂ the hypothesis that the defendant was at the crime scene and E a witness that observed the defendant at the crime scene. This corresponds to Thagard´s ‘principle of explanation’ (E2 sub b) discussed above (Section 2.2). Hypotheses that ‘together explain’ another proposition cohere.

There is always some uncertainty between two hypotheses in a chain, P(H₁|H₂) < 1 and P(H₂|H₁) < 1, otherwise there would be no point in modelling H₁ and H₂ as separate hypotheses. As Fig. 3 shows, the probability of H₂ given the evidence is therefore higher than the probability of H₁ given the evidence, P(H₂|E) > P(H₁|E).

In Fig. 3, the chain only contains two hypotheses (H₁ and H₂), but there are situations where the chain is longer, e.g. H₁—H₂—H₃ – H₄ – E. In this longer chain, ‘explaining E from H₁’ needs to move over H₂, H₃ and H₄. Since there is uncertainty between each link in the chain, the probability of H₁ given the evidence decreases when the number of hypotheses increases. This corresponds to Thagard’s ‘principle of simplicity’ (E2 sub c) discussed above (Section 2.2): ‘the more hypotheses needed to explain something, the lower the degree of coherence’.

Situations where two (or more) hypotheses together explain the evidence must be distinguished from situations where two hypotheses are negatively relevant for each other, and therefore incoherent. This is the case when H₁ and H₂ offer competing explanations for the same evidence. Instantiating H₁ will decrease the probability of H₂, and vice versa. This is known as ‘explaining away’ in Bayesian Networks. See Fig. 4.

As an example, H₁ could be the hypothesis that the defendant shot the victim, H₂ the hypothesis that the defendant has been framed by the police, and E a shoe print at the crime scene that matches the defendant’s shoe. This corresponds to Thagard´s ‘principle of competition’ (E6) discussed above (Section 2.2). Hypotheses that offer competing explanations are incoherent with each other.

We have now demonstrated that Thagard’s notion of coherence in terms of ‘symmetry’, ‘explanation’, ‘simplicity’, ‘contradiction’ and ‘competition’ can be translated into a probabilistic vocabulary. The only feature of Thagard’s definition of coherence that we have not translated is ‘analogy’.²⁶ According to Thagard (E3), ‘similar hypotheses that explain similar pieces of evidence cohere’ (Section 2.2). Unfortunately, Thagard does not explain what ‘similar’ stands for, and the analogy-feature of Thagard’s notion of coherence is therefore too unclear to be translated into probabilistic terms.²⁷

There is a debate among reductionists on how to measure the degree of coherence between two beliefs. Shogenji has proposed that the ratio P(A&B)/P(A)P(B) should be used as a measure (Shogenji 1999, 339). Other ways to measure the degree of coherence have been suggested by Olsson (2002), Fitelson (2003) and Douven and Meijs (2007). The question becomes increasingly complicated when we not only want to measure the degree of coherence between two beliefs, but also the degree of coherence in a set that contains a larger number of beliefs. The various ways to measure the degree of coherence proposed by Shogenji and others have spawned a debate on the merits and shortcomings of the proposed measures (Siebel 2005; Bovens and Hartmann 2005; Moretti and Akiba 2007; Roche 2013; Schippers 2014). Some arguments in this debate are directed at a specific measure. Several participants have argued that there is no objectively superior measure (Bovens and Hartmann 2005; Schippers 2014). This discussion will not be addressed in the present article.

Reductionism has provoked a fierce debate about the nature of coherence and its relation to probability. Some critics have objected that reductionism is an impossible endeavour, since coherence cannot be reduced to probability. We refer to this view as non-reductionism. Proponents of non-reductionism include Thagard (2004), Siebel (2005), Lycan (2012) and Amaya (2015).

no probabilistic function will ever be able to adequately capture coherence (Siebel 2005, 43)

There is a body of literature that attempts either to vindicate or to disparage less formal notions of coherence by, in one way or another, testing them against probability theory. […] The reason I take the demand to be unreasonable is that explanationists, at least in my case, take explanatory inference as fundamental. Every epistemology takes some norm or norms as epistemologically basic, and the result of the explanationalist’s wide reflective equilibrium (whether or not it convinces everyone) is that explanatory inference is basic. It is not derived from or justified by or to be tested against some more fundamental norm […] In particular probability theory is not to be accepted as a more fundamental norm. (Lycan 2012, 17–18)

According to non-reductionism, something important is lost when coherence is reduced to probability. We will take a closer look at non-reductionism in the next section and examine the arguments for this claim.

4. Siebel’s argument for non-reductionism

According to non-reductionism, something important is lost when coherence is reduced to probability. This begs the question: what exactly is lost? Most non-reductionists never answer this question (e.g. Lycan). An attempt to pin-point what is supposedly lost has been made by Mark Siebel. According to Siebel, coherence cannot be reduced to probability since probability cannot fulfil the role that coherence plays in explanations (Siebel 2011). That ‘H explains E’ says something more than P(E|H) > P(E).

explanation cannot be captured merely in probabilistic terms […] [and] this result has a devastating effect on … the project of developing a probabilistic theory of coherence […] if probabilistic accounts cannot cope with explanation, they will hardly be able to deal with coherence because … coherence is a function of explanation (Siebel 2011, 266)

Siebel gives two examples to demonstrate that this is the case. The first example is an argument where H consists of several propositions, including E. He observes that although P(E|H) > P(E), it is not the case that H explains E, since an inference that contains its conclusion as a premise lacks ‘explanatory value’ (Siebel 2011, 265).

Siebel is right that a circular argument fails to provide a reason for its conclusion, in addition to that what is already assumed in one of its premises, but this observation is misdirected as an argument against reductionism. It would have been appropriate if reductionism had stated that explanation can be reduced to probability, but reductionism makes no such claim. Reductionism only says that coherence can be reduced to probability. Reductionists think that positive relevance is a necessary feature of explanation, but they are not committed to the view that positive relevance is a sufficient feature of explanation. It is perfectly possible for a reductionist to say that in order for ‘H to explain E’ it is not sufficient that H is coherent with E (positively relevant for E). It must also be the case that H does not assume E as a premise. According to Peter Lipton, inference to the best explanation is well correlated with probability, but cannot be reduced to probability, as there are other things than positive relevance that contribute in making an explanation ‘lovely’, e.g. being simple and contrastive (Lipton 2004, 114). In the words of Frank Cabrera, some explanatory virtues are ‘confirmational’ and can be reduced to probability, but others are ‘informational’ and cannot be reduced to probability (Cabrera 2017, 1254).²⁸ There is another reason why ‘explanation cannot be captured merely in probabilistic terms’ (Siebel 2011, 266). Many explanations are causal and causality is a richer concept than probability. Therefore, explanation cannot be reduced to probability.

Siebel’s second example involves a scenario where a woman called Susan meets an untimely death.

Suppose Susan swallows a pound arsenic in order to commit suicide. Shortly after, however, she dies because she is run over by a bus. The probability of dying, given that one ingests a pound of arsenic is usually higher than the prior probability of dying. Nonetheless, it is not the arsenic but the collision with the bus which explains Susan’s death. (Siebel 2011, 264)

This argument suffers from the same problem as the first argument. Reductionism does not claim that explanation can be reduced to probability. Furthermore, it rests on an incorrect understanding of the probabilities involved. The point that Siebel wants to make with the scenario is that although the hypothesis that Susan swallowed arsenic (H₁) is positively relevant for the observation (E) that she is dead, it is not the case that H₁ explains E, given that she was run over by a bus (H₂). For this to be true, it is not sufficient that P(E|H₁) > P(E). It must also be the case that P(E|H₁, H₂) > P(E|H₂), and this is obviously incorrect. Since it is given in the scenario that Susan dies ‘because she is run over by a bus’ P(E|H₂) = 1. This means that P(E|H₁, H₂) = 1, and we can therefore conclude that P(E|H₁, H₂) = P(E|H₂).

5. Coherentist and probabilistic approaches

As was stated in the introduction, this article is about the question whether the concept of coherence, more in particular the concept as it is used in Van Koppen’s scenario theory and Amaya’s theory of inference to the most coherent explanation, can be reduced to probability.²⁹ It is not about the question whether coherentist approaches can be replaced by or reduced to (Bayesian) probabilistic approaches. In the ICS theory, coherence is not the only central notion; the concepts of causality and causal explanation also play a pivotal role. For the purpose of our article, it is important to keep those two questions apart. To claim that coherence can be fully explicated in terms of probability is not to claim that causality and causal explanations can be fully explicated in (Bayesian) probabilistic terms.³⁰

However, we want to briefly address the question what role coherentist guidelines can or should play in the assessment of evidence and proof in criminal cases next to probabilistic guidelines, in particular Bayes rule. In particular, we want to address the question whether the scenario theory and the ICE theory or their combination are only useful in the context of generation and pursuit, or whether they also can or should play a role in the context of justification. Stated differently, do coherentist guidelines only have a heuristic role in the generation and the pursuit of alternative scenarios or can or should they also play justificatory role in the evaluation and the final choice of the best scenario?

In the discussion between adherents of an ICS approach and those of Bayesian probability theory, there seems to be agreement that ICS is valuable or even a necessary heuristic tool to generate and develop different scenarios. The discussion revolves around the question whether ICS should also play a role in the context of justification, more specifically in the final comparison of scenarios and the choice of the best scenario.

First, however, it should be noted that the process of generation, pursuit and justification is iterative. For one thing, a lot of preliminary justification is given in the context of generation and pursuit, in particular when dealing with the question, which scenarios are sufficiently fruitful and reasonable in the light of the evidence and our background knowledge to give a first or further thought. Also, it is possible, and sometimes it is necessary, to return from the context of justification to the context of pursuit to do further investigations on one or more scenarios, or even go back to the context of generation to construct a new scenario.

Accordingly, the claim that IBE (or in the context of criminal law ICS) is (merely) a good heuristic is misleading to say the least. Until the very end of the investigations, an ICS approach can help us to sort out whether we have made mistakes in the construction of the base set or the contrast set, or whether we fell prey to one or more biases in assessing the coherence.³¹ A Bayesian perspective can also be of help to lay bare mistakes, possibly even better when it comes to certain biases such as the base rate fallacy or the prosecutor’s fallacy.

A second reason why ICS can play a role in the context of justification is that the Bayesian calculus lacks the explanatory considerations needed to answer the questions which evidence is relevant, what are the prior probabilities of hypotheses and what is the likelihood of the evidence given the hypotheses (Lipton 2001, 2004). It seems that in many cases, explanatory considerations are useful if not indispensable to help answer these questions.

The third and final observation we want to make about the relation between coherentist and probabilistic approaches concerns the fact that people seem to find it easier to reason in terms of coherence, epistemic virtues and causal explanations than in Bayesian terms.³² Accordingly, a coherence theory that 1) makes explicit the ways in which people implicitly reason and that 2) offers guidelines to reason in a more structured way and that helps them to keep track of whether they have taken all the necessary steps and that 3) formulates a checklist of often made mistakes and biases and that therewith helps to prevent them from falling prey to these mistakes, cannot only play a role in the context of generation and pursuit but also in the context of justification of evidential judgments in the context of criminal law, even if one would argue that—in the end—a coherentist assessment should be in accordance with a probabilistic assessment. Accordingly, the scenario theory, ICE and their combination seem to offer mid-level theories between intuitive, unstructured and ad hoc approaches and probabilistic, more specifically Bayesian, approaches to the justification of evidentiary judgments in the context of criminal law.

In the context of generation and pursuit, an ICS approach seems indispensable since neither the intuitive approach nor the Bayesian approach offer a heuristic to generate scenarios and to offer a quick global assessment of the fertility of a scenario. In the context of justification an ICS approach is useful to help determining priors, likelihood and the relevance of evidence. Moreover, in the context of justification, a mid-level coherentist theory has the advantage over the intuitive approach that it is much more structured and precise and it has the advantage over the Bayesian approach that it is more feasible.

If our observations are correct,³³ this would imply that IBE and Bayesian approach are not competitors. It seems that they are not merely compatible but complementary approaches, and that they are not merely complementary in the context of generation and pursuit but also in the context of justification.

Again, that is why we should keep the question about the reduction of the concept of coherence to probability apart from the question about the relation between an ICS and a Bayesian approach. Moreover, even if coherence can be reduced to probability, this does not imply that the concept should be eliminated. On the contrary, an explication of coherence in terms of probability can be fed back into coherentist approaches to create more precise criteria for the assessment of coherence.

6. Conclusions

Our collaboration has been fruitful, but we diverge in our conclusions, and have therefore decided to state our final thoughts separately.

Dahlman:

Our investigation corroborates reductionism. As we have seen, the coherence vocabulary can be translated into a Bayesian vocabulary by defining coherence as positive relevance. According to non-reductionism something essential is lost if coherence is reduced to probability. We have examined Mark Siebel’s argument for non-reductionism and found that it does not hold water.

The Bayesian vocabulary is more precise than the coherence vocabulary, but also more demanding. There might be situations in communication where the Bayesian vocabulary is too demanding, and the coherence vocabulary is preferable. However, for an exact evaluation of evidence, the Bayesian approach is superior.

Mackor:

Our investigation has offered an explication of coherence of factual beliefs in terms of probability. Regardless of whether reduction is possible,³⁴ the explication of coherence in terms of probability is fruitful if only because it forces adherents of the coherence approach to clarify not only their concept of coherence but also the nature of inference to the most coherent explanation or scenario.

An issue that needs further exploration is whether epistemic virtues can be of use in the assessment of scenario’s and whether these virtues can be explicated in terms of coherence and probability. In Section 3, we offered an explication of the virtue of simplicity (Thagard’s principle E2 sub c), but other virtues should be analysed too.

We have argued that one should separate the question about reduction of coherence to probability from the question about the relation between coherentist and probabilistic (Bayesian) approaches. Regardless of whether reduction of coherence to probability is possible, we agree that an ‘inference to the most coherent explanation’ theory could fulfill an important function both as a theory for the generation and pursuit of scenarios and as a mid-level theory for their justification.

Footnotes

References