Fishers of Evidence Gets Confused about Math

Someone asked me about a confusing critique of the math in On the Historicity of Jesus by a YouTuber who goes by the moniker Fishers of Evidence. I don’t know his alignment in the debate. But he has posted a short eight minute video entitled The Error of Richard Carrier. And this is a good opportunity to teach some simple math you can use in your own probability reasoning.

The central gist of FoE’s argument isn’t well explained, and someone who doesn’t know math won’t know what he’s talking about. Even someone who knows math will be confused, because he confuses a variety of mathematical concepts, such as conflating odds with decimal probabilities. But to steelman his argument as best I can, he means to say this:

When you are going to multiply the probabilities of individual events, in order to determine the probability of the conjunction of all of those events, you can’t just multiply their individual probabilities if those events are in any way dependent on each other.

For example, the independent probability that I am rich may be 1 in 100. The independent probability that I am the winner of a multimillion dollar lottery may be 1 in 1,000,000. But the probability I am both is not 1/100 x 1/1,000,000. Because once I’ve won a multimillion dollar lottery, the probability that I am rich is no longer that independent base rate of 1 in 100. It’s now, let’s say, 2 in 3 (e.g., if only 1 in 3 such persons squander their millions and become poor again, all the rest will be rich, which means 2 in 3 will be, not 1 in 100 of them). And if it’s already known and confirmed that I won such a lottery, then the probability that I am rich is not 2/3 x 1,000,000 either, but in fact essentially just 2 in 3. Because if we disregard the tiny chance we are mistaken that I won such a lottery, the odds I won, given all the evidence I did, is 1/1, and so we get 2/3 x 1/1 for…well, 2/3.

Notice this is still all multiplying. That’s important. Because not knowing how to do this correctly with odds ratios, is the Error of FoE. He thinks dependent probabilities can’t be multiplied. He must be high. Because that’s the only way they teach it in math class. What changes is not multiplying, but what probability you put into the multiplication.

Relevance to OHJ

The best way to fix up FoE’s argument is to say this:

When I count up all the evidence bearing on the historicity of Jesus, I multiply a string of probabilities in OHJ, in the form of likelihood ratios, one for each sub-division of evidence, to get a final probability (a final odds ratio) of all that evidence being that way. Which is therefore a probability of that conjunction of evidence. This is to make it easier for critics to examine and challenge or consider each step of reasoning that goes into the conclusion. But FoE claims that in doing this, I am making the same mistake as calculating the probability that I’m both rich and a lottery winner by multiplying 1/100 by 1/1,000,000, when really it should be 2/3 x 1/1 and thus in fact just 2/3. He doesn’t use that example. But it gives us a clearer form of his argument than he presents.

Importantly, he never shows this is the case. Ever. At no point in his video does he ever mention any ratio I assign to any item of evidence, nor does he ever explain how any particular one of those ratios should be changed to account for any dependence effect. This is lazy criticism. As I’ll explain below, there is actually a really good way to test and critique OHJ here, if only a critic actually did it, rather than assume, as if by magic, that any of my ratios is incorrectly assigned, without examining a single one of them. But before I get you there, let’s first circle back and tour his video a bit to see what’s going wrong here.

Did He Just Say That?

FoE correctly describes Bayes’ Theorem, but only in the short form, which is an incomplete formula to a lay observer, who won’t know what’s hidden in it (see Proving History, pp. 69 and 283-84). He says (and agrees) that the probability of historicity given the evidence equals the probability of that evidence given historicity, times the [prior/general] probability of historicity, all divided by the probability of the evidence “whether or not” Jesus existed. He makes an erroneous statement at this point, at timestamp 3:30, when he says, “which of course equals one because we have the evidence that we have.” That’s not how that works. The P(e) that completes the denominator of the short form is not simply “1” (100%) because e “is what we have.” That’s the Fallacy of Foregone Conclusions.

And he seems to know this, so either he misspoke or he is badly confused when applying his own examples to the question of historicity. In his own example of how this works, regarding meningitis, he gets it right. When M is “I have meningitis” and e, the evidence, is a positive test result, then P(M|E) = P(E|M) x P(M), divided by P(E| “whether M or not”). Just P(M) means the base rate, the probability before I get tested, of people like me having meningitis. But P(E| “whether M or not”) is not 1. If it were, that would mean everyone who gets tested for meningitis always gets a positive result, whether they had meningitis or not! That would render such a test completely useless.

And accordingly, FoE, despite saying P(e) is always 1 because “that’s the evidence we have,” correctly shows on screen that this is not true for the meningitis testing. He posits for his example that the probability of e, “a positive test result,” is 0.5% when you don’t have meningitis (aka ~M) and 99.5% if you do have meningitis (aka M); and he posits a base rate of having M of 1 in 1000, which means a prior probability of 0.001, or 0.1%, one tenth of one percent. And that means P(e), the probability of a positive test result “whether or not” you have M, is not 100% but in fact [P(“positive test result for M”|M) x P(base rate of M)] + [P(“positive test result for M”|~M) x P(base rate of ~M)], or, as he correctly shows on screen, (0.995 x 0.001) + (0.005 x 0.999) = (0.000995) + (0.004995) = 0.00599 (or about 0.6%). Which is nowhere near 1 (aka 100%).

One therefore would never say the probability of a positive test result “whether or not you have meningitis” is 100%. Because, in his own example, it’s 0.6%! And FoE seems to know this, as that’s what he shows on the screen. But he doesn’t connect the two examples, so he never notices his mistake in saying P(e) equals 1. The really weird thing here is that if he really thinks P(e), the whole denominator of every Bayesian equation, is always 1 because we always “have the evidence we have,” then you don’t need the denominator at all. The probability of anything is then just P(e|h) x P(h). That should have clued him in that he was making an error in his statement here. I will charitably assume he misspoke and didn’t really mean to say that.

Otherwise, he is confusing two completely different probabilities, and to help anyone else from making that mistake even if he didn’t mean to, remember this:

The probability that the evidence exists given that we are observing it, and the probability that that evidence would exist given that a particular event happened in the past, are not the same probability.

So, for example, if assessing the evidence of a murder, FoE found blood on the accused, he could rightly say “the probability that the accused is bloody, given that I observed and verified the accused is bloody” is 1 (or near enough; there is always some nonzero probability of still being in error about that, but ideally it will be so small a probability we can ignore it). But that doesn’t answer how the blood got there. What we want to know is: What is the probability that the accused would be bloody given that they murdered the victim? And then, what is the probability that the accused would be bloody (= that they will test positive for meningitis / that the accounts of Jesus we have would be written when and as we have them) whether or not they murdered the victim (= whether or not they have meningitis / whether or not Jesus existed)?

That is not going to be 1. The blood could be their own; it doesn’t follow that the blood is from the victim. Or the blood could be there because they tried to rescue the victim, not because they murdered them. It doesn’t even follow that every time someone murders someone, they get or keep the victim’s blood on them. Like a positive meningitis test, many people test positive, whether or not they have meningitis. Moreover, many test negative, whether or not they have it. Similarly, many a biography is written of men, whether or not those men existed. So P(e) is frequently not 1. And in fact whenever it is 1, that means there is no evidence for the hypothesis at all.

You heard me right.

Just as if everyone tested positive for meningitis: that test would give us no evidence whatever whether you had meningitis or not. All we would then have to go on is the base rate. The test contributed nothing. Similarly, if the evidence for Jesus is 100% expected “whether or not Jesus existed,” then there is no evidence for Jesus. All we have to go on then is the base rate. The “evidence” in that case tells us nothing at all about whether he existed or not. Though of course P(e) can never be 1, when P(e|h) is not. P(e) can only ever be 1 when not only P(e|h) is 1, but also P(e|~h) is 1. So a P(e) of 1 in his meningitis example would be impossible anyway, since he assigned a false negative rate of 0.005, which entails a P(E|M) of 0.995, making that the upper limit for P(e), the probability of a positive test “whether or not one has meningitis.” (Try it. Play around with the numbers and see if you can get higher. Remember, P(e) = [P(e|h) x P(h)] + [P(e|~h) x P(~h)], and always P(~h) = 1 – P(h).)

So either FoE means there is no evidence for Jesus. Or he screwed up his math. Or he didn’t mean to say P(e) in the case of Jesus “of course equals one because we have the evidence that we have.” I’ll charitably assume the latter.

Getting to the Point

At timestamp 4:33 Fishers of Evidence says something about generating a prior probability from the Rank-Raglan data, but presses no criticism there I could discern, unless when he says a prior must be generated “without reference to these bodies of evidence” he means the RR data comes circularly from the evidence I put in e, or he means a prior can’t be based on evidence. Both would be incorrect. A prior is always based on evidence—everything you put in b. And you can put anything in b that you exclude from e (and vice versa). As I explicitly state in OHJ, I put the RR data into b, and then properly excluded that data when I examined the remaining content of the Gospels (see my discussion of this mistake when made by Tim Hendrix). But since FoE developed no clear criticism from this, I’ll move on to his central point…

At timestamp 4:55, FoE starts on about the difference between dependent and independent probabilities. He confusingly frames this in terms of summing vs. multiplying probabilities. But he never shows what he means. To understand what he means, see my better steelman summary above. Although he is confusing probability equations using odds ratios and probability equations using percentages. Though even using percentages, you can multiply dependent probabilities. You just have to make sure you are doing it correctly (see my previous discussion of this).

For example, the lottery example: if we didn’t know whether I had won a lottery (or whether I was rich, either), the probability that I’d be both is then indeed 2/3 x 1/1,000,000 (not 2/3 x 1/1; nor, either, 1/100 x 1/1,000,000). In percentage form, that’s (roughly) 0.67 x 0.000001. In notation: P(rich & lottery) = P(rich|lottery) x P(lottery). Note that multiplying is perfectly fine. As long as I multiply P(lottery) by P(rich|lottery) and not P(rich). Because P(rich|lottery) is 2/3, while P(rich) is 1/100. The latter would be incorrect if we intended to derive the total probability by multiplication. But we can still derive the total probability by multiplication. You just have to use the correct probability. FoE seems not to know this.

FoE thus commits two errors here. First, he incorrectly claims you can’t derive a total probability from a series of dependent probabilities by multiplication. To the contrary, that is in fact how you usually do it. And second, he never identifies any examples of dependent probabilities in my series of estimates in OHJ. He correctly notes that there are causal connections between groups of evidence, e.g. the Epistles had causal effects on the Gospels, and the Gospels had causal effects on Acts and the Extrabiblical evidence. But he doesn’t identify a single probability I produce that is affected by these causal relations. For example, in Acts, I don’t just come up with “a probability of Acts given historicity/ahistoricity.” I actually divide Acts into three pieces of evidence, none of which reflect the dependency relations FoE alleges, except one, and for that I give a dependent probability, exactly as required by the math (see below). In other words, it does not appear FoE actually examined any of my estimates for any actual dependence relations. If he did, he would have discovered (a) many don’t have such relations to account for and (b) the ones that do, I already accounted for!

For example, Acts is dependent on the Gospels only for the material it shares with the Gospels, which I don’t explicitly assign any probability to! I assume that content is 100% expected on either theory, because it’s 100% expected given “the Gospels exist,” and the Gospels exist, regardless of whether Jesus did. I assign it odds of 1/1. In other words, no effect. That’s a dependent probability, like the 2/3 chance of being rich given having won a lottery. FoE seems not to have noticed. His criticism is thus completely inapplicable here.

Meanwhile, for material pertaining to the history of the church not recorded in the Gospels, Acts is independent of the Gospels. And that’s the only material I assign a probability to. Although that material is causally dependent on the Epistles, I already account for that in my estimations of the likelihoods of those contents of Acts given what’s in the Epistles. Indeed, the fact that Acts definitely used the Epistles as a source entails it deliberately contradicts the Epistles when indeed it frequently does so, which entails the probability that Acts is a reliable source is near zero. Although that isn’t a probability I use in OHJ. I only estimate how likely it is that Acts would lack the things it does, even given the fact that its author knew and was influenced by (and aiming even to contradict) the Epistles. So the probabilities I put in are already dependent probabilities as required. FoE’s criticism is again wholly inapplicable.

And so on throughout.

It’s especially weird that FoE doesn’t know you can derive total probabilities with dependent probabilities by multiplication. Because he briefly references independent probability calculations in gambling as an example of doing it wrong, and even shows a dependent probability calculation in there: correctly, as a multiplication. Though he doesn’t discuss this. So what is going on here? Either he doesn’t really know how to do that math, or he is actually trying to deceive his viewers by omitting his explanation of it, since that would refute his entire claim that I’m supposed to add and not multiply (a concept he never illustrates with any example).

Take poker. Suppose we are asking about the conjunction of two events, drawing a king from a full poker deck and immediately drawing a second king from that same deck. The probability is not, indeed, as FoE implies, 4/52 x 4/52 (there being 4 kings in a deck and 52 cards in a deck), because the second probability is indeed a dependent probability, the very kind of thing FoE is talking about: the second probability is dependent on what was drawn first. If the first draw was a king, then the probability of two kings drawn is 4/52 x 3/51 (since now, after the first king is drawn, there are only 3 kings and 51 cards), which is 12/2652, or 0.0045 (rounded), aka 0.45%, or a nearly half of one percent chance. Notice we are still multiplying, not adding. Contrary to what Fishers of Evidence claims. And we can do this for the evidence of Jesus as well. And that’s exactly what I do. All throughout OHJ.

For example, what is the probability Acts would have faked up the trial speeches for Paul to match the street sermons, and thus included references to a historical Jesus, instead of as we have it now, trial speeches that bizarrely omit any references to a historical Jesus, and street sermons that include such references? This would be, one might argue, a probability dependent on the existence of Luke’s Gospel. In other words, Luke certainly knows the material that evinces a historical Jesus, and it could have caused him to fake evidence everywhere in Acts. But in this case, he didn’t. He kept this bizarre incongruity between Paul’s trial speeches and street sermons (OHJ, pp. 375-80). That fact by itself is more likely if Jesus didn’t exist than if he did—though as I argue, the difference is extremely small: a likelihood ratio of only 9/10 against historicity on the a fortiori side. And that’s what that ratio would be even considering the fact that Acts was written by someone who knew the Gospel material. In other words, this 9/10 is the same figure as the 3/51 in the poker example above, and the 2/3 in the lottery example. It is thus not invalid. It’s perfectly correct. FoE’s argument has no relevance here.

My taking this dependence into account is even explicit: I estimate that even if Jesus didn’t exist, the fact that Luke would add a reference to historicity in Stephen’s speech given that Luke wrote the Gospel under his name is just as likely as that such a reference would be there if Jesus did exist. So the ratio is 1/1. Which means it has no effect. Stephen’s speech argues neither for nor against historicity. As I wrote: “when Luke inserts into Stephen’s speech a brief reference to the historicity of Jesus…this could obviously be Luke importing his own narrative assumptions,” among other possibilities I enumerate (OHJ, p. 383). Thus, I’m fully taking into account the fact that this is a dependent probability. And my incorporation of it is mathematically correct. I don’t literally assign either outcome a 100% probability (see OHJ, p. 605; cf. pp. 288-89, n. 18 and p. 357, n. 122); it could be 50% likely Luke would fake evidence and 50% likely he’d include a reference if Jesus really existed. What I estimate is that whatever those two probabilities may be, they are going to be so nearly the same that there is no measurable difference between them, and therefore Stephen’s speech argues for neither theory. And this is so even given Acts’ reliance on the Gospels and Epistles. So their being dependent probabilities makes no difference to the math. Contrary to what FoE claims.

I similarly adduce that the Gospels, deriving material from the Epistles (as I explicitly admit they do in OHJ), have the same probability of including Epistle-connected material whether Jesus existed or not, because of their dependence on the Epistles for that material. In other words, the Gospels cannot corroborate that material, if they are not independent of it. Thus, again, I am doing exactly what Fishers of Evidence says I should: accounting for the interdependence of some of the evidence when I assign probabilities.

All my probabilities thus are just like the poker draw probabilities: they are already factoring in the causal relations among the different categories of evidence. This is indeed so explicit in my discussion of the extrabiblical evidence (in Ch. 8) that it should have been obvious to FoE that that is what I was doing. My finding that no external sources corroborate the Gospels is derived from their dependence on those Gospels. Thus, that Tacitus should mention a Gospel claim about Jesus (if in fact he ever did) is already 100% expected on the existence of the Gospels, regardless of whether Jesus existed or not. That reference in Tacitus thus has no effect on our final probability of historicity. That’s how dependent probability works. And ironically, here it’s Christian apologists who typically don’t grasp the point that Fishers of Evidence is making: that the probability the extrabiblical sources would mention Jesus, even if he didn’t exist, is dependent on the Gospels having already done so (and their Christian informants subsequently relying on the Gospels, as we know they did).

If Fishers of Evidence thinks I have mis-estimated a conditional probability anywhere in OHJ, he should explain which probability, and what the correct value of it is and why. In other words, if dependence changes anything, he needs to identify what it changes, and by how much. You can’t just say it will change something. Because you don’t know. In any given case, I might already be factoring in the dependence, just as I have demonstrated I did in several cases here, and just as I did here when producing the multiplication (not addition) that correctly predicts the probability of drawing a pair of kings straight off a full poker deck.

So, am I factoring in dependence correctly, or am I not? If you think not, show me where, and why you think I’ve inadequately accounted for the effect of dependence. If you can’t show me where, then you don’t know I did. Therefore, you can’t claim I did.

Fishers of Evidence never shows I did. Anywhere. He shows no sign in fact of even knowing I did. Or of even checking. It doesn’t even seem apparent to him that he is supposed to check.

How Did He Figure That?

FoE spends some time on the famous Roy Meadow example of failing to correctly use the dependent rather than the independent probability in his evidence calculations. But this has no relevant parallel in OHJ. And Fishers of Evidence shows no parallel. If he could, that would be something, and indeed something I’d most like to see found, if any such effect is there.

But moving on to his conclusion, at timestamp 8:40, Fishers of Evidence says he recalculated my result by using a sums method. Or more precisely:

I have recalculated Carrier’s probabilities arithmetically. There are different ways of doing this, by using different weightings for example. But I have simply weighted his groups equally. My arithmetic calculation gives an upper bound probability of historicity of 46% and a lower bound of 18%.

He doesn’t show his calculation or even explain what he means. How he derived these numbers is mysterious. I try to imagine how he’d do it with a poker deck, and can’t think of any way to “arithmetically” get any result other than the same one anyone would get with multiplication. So this sounds like taking that correct 4/52 x 3/51 calculation and “redoing” it somehow to get a different result: whatever he is doing, it can’t be mathematically valid.

At best he must be assuming—without ever confirming or observing or demonstrating the fact—that I didn’t take into account dependence when I assigned my probabilities in each category—even though I explicitly state I am assuming those dependencies, and even rely on that fact repeatedly to get the results I do (like the 1/1 result for Stephen’s speech). He seems to think I was doing 4/52 x 4/52. But he never explains where I do that. And I don’t. I’m doing 4/52 x 3/51. Every single time. My math is correct. I have no idea what his math is. As he doesn’t show his math, there’s no way to be sure what he’s doing wrong. He claims to be summing weighted probabilities, but I used ratios, not probabilities in the way he describes.

Indeed, as best I can tell, whatever his alternative method, he says he assumed an equal weighting for every item of evidence’s dependency. Which implausibly assumes the effect of every item of evidence on every other (such as the weirdness of the trial speeches in Acts on the omission of James from the entire public history) is exactly the same. He just assumes that. Without argument. But even before attempting an argument, I can already tell you, it would be cosmically bizarre for that to actually be the case. And it’s clear FoE has no argument for it being the case, that he did not even consider the matter. Which renders his alternative result completely meaningless. It has no connection with any argument for dependence or causality among the items of evidence I demarcated. His own result is thus, in fact, far worse than even he thinks mine is.

Using Acts to Illustrate the Point

Let’s talk about how mathematicians actually combine dependent probabilities. And what I actually did in OHJ.

I’ve already discussed above how I already make estimates of dependent probabilities in OHJ with respect to Acts. More specifically, I break that down into three things: the missing family (and other personages who should have had a causal effect on the history of the church); the oddness of the trial speeches (which read as if there were no historical but only a cosmic Jesus); and the rest of the content of Acts (OHJ, p. 386). I find that the rest of the content of Acts is just as expected on either historicity or not, and thus assign a ratio of 1/1. This is actually already a dependent probability: because Luke is being caused by the Gospels to repeat Gospel-derived historicity material, the dependent probability that he would do so even if Jesus didn’t exist is 1 (at least, as near to one as makes all odds), once we grant the Gospels causally influenced him. That’s exactly like realizing the probability of drawing a second king is 3/51 and not 4/52. There is no other way to do this math. Any other method he uses, if it gets him a different result than multiplying 4/52 by 3/51 in the poker case, he’s doing it wrong. And likewise if he makes the same mistake with my assignment of 1/1 here.

But since 1/1 multiplied by anything has no effect, we can ignore that now and look at the other two ratios. For those other two items in Acts I assign on the a fortiori side a likelihood of 4/5 for the missing people and 9/10 for the weird trial speeches. (Which are already super weak evidence ratios, BTW…good evidence should weigh 1/4 or 4/1 or more or even a million or a billion to one, so I am not making anything like a strong claim of effect here. But moving back to the point at hand…) For FoE to claim that these should not be multiplied against each other (4/5 x 9/10), he needs to show two things: that those two things are dependent on each other; and that that dependence changes the ratios to something other than I assigned. For example, imagine Luke’s not thinking to ever mention James ever led the Church, in a supposedly researched history of that Church, is somehow caused by his trial speeches also not seeming to know about a historical Jesus, such that wherever there is the latter, there will always be the former. In that case, the dependent probability of the omission of James is 1/1, and it no longer has any effect. We are then left with simply the 9/10 oddness of the trial speeches. Just as if drawing a king from the deck, at odds of 4/52, causes the next draw always to be a king, such that the probability of drawing two kings right off the top is simply 4/52, not 4/52 x 3/51.

Unfortunately, Fishers of Evidence never does this. For anything in OHJ. He doesn’t explain how, for example, the trial speeches omitting a historical Jesus, affects the probability of James being omitted from the entire history of the church. In what way are they dependent on each other? And even if we can come up with any plausible dependence (and they have to be plausible; wildly implausible dependencies have too low a probability to show up in the math at the resolution I employ), does it change my estimates at all? Is it really credible to say the weirdness of the trial speeches would always cause an omission of James? Certainly not. If they have any effect on each other at all, it’s extremely small. There are too many ways either could happen independently of the other. So is that tiny effect large enough to make the conjunction of both together any different from 4/5 x 9/10? A difference, that is, that shows up at that resolution? (For example, a difference of a millionth would be so small it would disappear when rounding to the nearest whole percentage point.) Such causal dependencies thus have no visible effect. (And even if they do, the effect may be in entirely the other direction: see comment.)

So if Fishers of Evidence wants to say it would have a visible effect, he has to show how it changes that conjunction’s probability (or the probability of any actual conjunction I score in OHJ), and to what—that is, what it changes the probability to. I fully welcome such revisions. If anyone can show that the probabilities I assign in OHJ should be different because of causal connections I overlooked or wrongly estimated the effects of, that would be progress, and well worth issuing a corrected edition. But you have to show that, before you can claim any such error exists. Otherwise, my 4/5 for the omission of Jesus’s family in Acts given the oddness, at the same time, of the trial speeches in Acts, is as valid as that 3/51 in the poker example. Fishers of Evidence has not shown otherwise, for any ratio I propose, anywhere in OHJ.

Conclusion

Fishers of Evidence also throws in at the end the completely undefended claim that, even apart from the dependency issue he thinks I didn’t account for (even though I did), my probability assignments were arbitrary. He doesn’t explain why they are arbitrary. The cases I make for them are far from arbitrary. One should be able to refute those cases and make a case for a different estimate if in fact they were arbitrary. So calling them arbitrary is just a lame way to avoid having to address any of the arguments and evidence presented for every single estimate I include. That’s not rational argument. That’s hiding from rational argument. It’s like telling yourself the sky isn’t blue and hoping that mantra convinces you so you never have to actually argue the point.

But the overall gist of his video is culminated at timestamp 10:06, where he claims weighing evidence is always a matter of adding, not multiplying, probabilities. That’s false, as my example for the poker deck shows: dependent probabilities are still multiplied, not added. He seems not to understand the fact that when using the Odds Form, we are dealing with ratios, which are always multiplied, never added. As with drawing two kings at poker: the causal effect of drawing one king, is reflected not by addition, but by changing the probability multiplied in. It’s not 4/52 x 4/52, yes. But it is 4/52 x 3/51. It’s still multiplication.

Apart from that error, if FoE thinks the causal relationship between any two pieces of evidence I assign likelihood ratios to would make the outcome different than I estimate, he has to show what the new multiplication is, not play around with addition. For example, if I said the odds of drawing two kings straight off were 4/52 x 4/52, it would be mathematically false to say I’m wrong because I need to add, not multiply. A mathematically correct criticism would agree we need to multiply, but that the dependence effect changes the second multiplier from 4/52 to 3/51, and that the correct result comes not from any addition, but from multiplying the correct ratios: 4/52 and 3/51. So what ratios does he propose we should multiply, other than mine? He never once says. Nor does he give any reason to believe the ones I propose are incorrect. Even granting every plausible dependence among every item of evidence in my array, they may all be just as correct as the 3/51 we must multiply by 4/52 to get the correct result at poker. It’s quite clear he doesn’t understand that, and has nothing to say about the actual, correct way to correct my math—if anything in my math is incorrect.

But indeed, if anyone can find a dependence relation between any of my probabilities that I incorrectly or don’t account for, and it’s enough to make at least a percentile of difference in the final result, let’s hear about that! Because that could be an important correction needed to OHJ. But until you can show that, you don’t have any criticism to make here.

7 comments

  1. FoE is correctly doing a calculation of a dependent probability at 5:22:
    “The chance of drawing two hearts from a deck of cards is 13/52 times 12/51”

    I can’t make any sense of the claim that probabilities should be added either, I can’t imagine a way to make this valid, but I’d like to see FoE’s calculations.

    I also think he is generally confused when it comes to dependent probabilities, he seems to believe that any sort of connection between two pieces of evidence would make them statistically dependent, which is not generally true.

    Reply
    1. Good catch. I missed his on-screen showing of a dependent probability multiplication. He doesn’t discuss it as such, so I overlooked it. I’ve emended my article to reflect this point. I’m astonished. Did he not know he did that?

      And yeah: addition is typically only for summing probabilities with the equivalent of an “or” operator (either x happened or z happened; what then is the probability of y given either x or z), which is exactly what the denominator of a standard Bayesian equation is. Addition is not used for calculating the probability of the equivalent of an “and” operator (a conjunction of events). Sure, all multiplication is the iteration of addition (e.g. 1/2 x 1/3 = 1/6 because 1/(2+2+2)), but that’s the same as multiplication. So he shouldn’t be getting a different result with it.

  2. Note: It just occurs to me there is another point I should make.

    In the paragraph where I discuss the point, “Is it really credible to say the weirdness of the trial speeches would always cause an omission of James?” and I answer “Certainly not,” one could propose a scenario whereby Acts is using a source (let’s say, a pre-Acts history of the church) that omits any knowledge of a historical Jesus (hence simultaneously causing an omission of James from the events of the church and the omission of an earthly Jesus from the trial speeches of Paul). That would get one of those to be 100% expected on the occurrence of the other.

    However, by positing that scenario, you just changed the probability substantially—in the other direction. Because the likelihood now that the preceding history of the church used by Acts lacked a historical Jesus, given that Jesus existed, is far less than 9/10. Indeed, it’s well below 1/4, even a fortiori. In other words, if Acts added a historical Jesus to an earlier history that lacked one, it is all but certain Jesus never existed, and a historical Jesus was later fabricated—by authors doing exactly what this scenario entails Luke did.

    So you need to be careful in what you are proposing to create dependent probabilities among the evidence.

    If you want to tie two pieces of evidence together by some hypothesis of how they both came to match the appearance of a “no Jesus” result, that hypothesis cannot be presumed without substantially reducing the probability Jesus existed.

    It’s a Catch-22 for any historicity apologist. What you need is a scenario whereby that conjunction is expected even if Jesus existed. But then you have to compare the prior probability of that scenario, with the prior probability of the alternative (e.g. a pre-Acts that lacked a historical Jesus). The latter might actually be more likely. After all, how would some proposed lost evidence of a historical Jesus simultaneously cause Acts to omit both James (from history) and a historical Jesus (from Paul’s trial speeches)? Far more likely that would be caused by evidence against the historicity of Jesus, not by evidence for a historical Jesus. A pre-Acts that omits a historical Jesus would easily explain that conjunction. But you would have to come up with a pretty convoluted theory to get, say, a pre-Acts that includes a historical Jesus yet still causes him to disappear in our Acts at precisely those two points (the role and existence of James altogether, and the content of Paul’s trial speeches).

    And here is where addition comes in: since the probability that either kind of pre-Acts existed is not 100%, the probability of each existing must be multiplied by the probability of the resulting conjunction of evidence, and then those two products added together, to get what you need for the final Bayesian calculation. The end result is not likely to favor historicity. At least, not any more than my estimates already do. And to gainsay that, you have to actually do the math. Correctly. Otherwise you can’t claim to know it will come out differently.

    Reply
  3. Patrick Mitchell March 22, 2017, 5:37 pm

    Oh Really, Dr Carrier [the following text also posted here]

    In mathematics, we often take common words and give them highly specific meanings consistent with their use in mathematical reasoning with its high degree of rigour compared to other subjects.

    Dependent, independent, hard assumption, softer assumption, etc. are such words. When we talk about independence being a hard assumption in probability theory we mean that probabilities are provably independent of each other. Provable independence is mainly restricted to theoretical models, such as the simple dice and playing card models of basic probability theory. When we set up these models we specify rules such as the dice are not loaded and the result a dice roll has no influence on subsequent rolls, and card decks are well shuffled.

    So if we draw two cards from a deck, then the probability that the first card is a King is 4/52. The probability that the second card is a King is dependent on what the first card was. If the first card was a King it is 3/51. It was not it is 4/51. The probability of drawing two kings without adjusting for the first card is strictly 4/52×3/51. The probability of drawing a King is only a question of how many kings there are in the deck divided by the total number of cards. We are tacitly assuming that there is no other influence. This assumption may be wrong. It may be a deck of kings. But it is an assumption that we make and if that assumption is wrong we’re going to be way out with our probabilities.

    In the real life forms of these models, when highly improbable events happen a natural reaction is to suspect that the rule of independence doesn’t apply. For example, if we roll six consecutive sixes we will suspect that the dice is loaded. If we draw a highly improbable hand from a deck of cards we will suspect that it has not been shuffled properly. Going beyond these into the more indeterminate situations of healthcare, psychology, economics or history, it is rare for independence of probabilistic variables to be provable. In certain situations we do assume independence. Then great care is taken to assess probabilities objectively and there is good reason to believe results are independent. An example is football pools where the results of simultaneous games are modelled, and geometric combinations are used. Often, though, when geometric calculations produce extreme probabilities we suspect a key assumption is not met. Roy Meadow’s 73,000,000 to 1 chance was suspiciously low and similarly the lower bound of Carrier’s probability range of 0.12% is also suspiciously low.

    In reality we approach claims based on geometric probability combinations with suspicion unless there is positive reason to believe that there is no third factor that influences the probabilities we are considering similarly. In other words, we have to be confident that the dice is not loaded and that the cards are well shuffled, and in Roy Meadow’s case that cot death does not run in families, and in Carrier’s case that there is no factor influencing the probability estimates that trends them in the same direction.

    For most real-life situations we cannot assume independence and therefore non-independence becomes the default position which we assume unless there is a compelling reason not to.

    Can dependent probabilities be combined geometrically?

    Absolutely. In the card example above the probability of drawing a second King is dependent on whether the first card drawn was a King and this can be fairly accommodated using geometric combination. But it depends on knowing the dependency exactly. It becomes dangerous when the dependence is not known exactly as in the case of Roy Meadow where the evidence didn’t seem to suggest that cot death ran families, but to assume that was a major error. The case of Jesus is totally different from the dice and gambling models of probability theory. In those we can calculate theoretical probabilities exactly. We just can’t do that with Jesus.

    Dr Carrier’s probabilities are not like poker draw probabilities. Those probabilities are deterministically defined in the rules of the game and can be calculated exactly. Everyone will come up with the same result. This is not true of the probabilities relating to Jesus.

    This point is readily illustrated from the current debate. Johan Rönnblom agrees with me that the chance of drawing two hearts from a deck of cards is 13/52×12/51. Of course he agrees with me. It’s true and anybody examining the same problem will come up with the same result. This one is solid enough to use in geometric probability combinations.

    On the other hand in Richard Carrier’s comment of 19 March 2017 11:55 am second paragraph he says. “Because the likelihood now that the preceding history of the church used by Acts lacked a historical Jesus, given that Jesus existed, is far less than 9/10. Indeed, it is well below 1/4”. Is that so? Is it below 1/5 or 1/6. What is it? 1/10? 1/8? And if one why not the other? And do all observers agree on this? Are assertions of this kind strong enough to use in geometric probability models? I think not.

    The reason that this is so important is that for geometric probability combinations, any errors in probability estimates will be multiplied and have a large effect on the outcome. If we estimate the chance of rolling a 6 as 1/6 when in reality for the dice at hand it is 1/5, our probability of rolling 6 sixes will be out by a factor of 3. This means that we must know the probabilities with a high degree of accuracy in order to be confident in the result. Relatively small errors in the probabilities we estimate, that we are unaware of, can have a large bearing on the result as was the case for Roy Meadow.

    So in probability the bar is high for the accuracy of probability estimates as it is for independence. So high it can never be met with a topic such as historicity v mythicism and that means that geometric probability models should not be used.

    The error that Carrier made in his books, and his rebuttal to my video, both indicate that he doesn’t get it. He is using the term independent in the way that a historian would not a mathematician. The lack of objectivity in his probability estimates, his attempt to arbitrarily compensate for what he sees as dependency, and his failure to appreciate that independence can never be established to the satisfaction of the mathematical definition for this kind of problem in history, all point to his not understanding what mathematicians mean by the term.

    Are Carrier’s probability estimates arbitrary?

    This is not a criticism but an inevitable consequence of the system under study. It is not possible to calculate precise probabilities and the estimates used by Carrier necessarily involve a degree of judgement. That means the estimates will vary from one person to another and that means that they are arbitrary. That is what the word means. If a staunch historicist addressed the same data, it is possible that they would revise their views and become a mythicist but it is also possible that they would come up with different probabilities.

    Carrier asks:

    “So, am I factoring in dependence correctly, or am I not? If you think not, show me where, and why you think I’ve inadequately accounted for the effect of dependence. If you can’t show me where, then you don’t know I did. Therefore, you can’t claim I did.

    Carrier’s factoring of dependence is no worse than anyone else’s would be, his error is not that he has done a poor job of it. His error is failing to appreciate that no one, not he or I or anyone else, can do it with the rigor necessary to satisfy the assumptions required to justify geometric probability models.

    Carrier’s model is simply too vulnerable to factors that he and in particular his readers doesn’t know about. He has not made any attempt to address even the usual suspects. For example, given that the probability estimates required a degree of judgement, and the same person made these judgements, how do we know that these judgements were not being systematically influenced towards historicity or mysticism? Further, was an effective Chinese Wall system employed between the probability estimates and the final output of the model? Those are modern unknowns. There are ancient ones too. Who amongst ancient people believed historicity, who didn’t believe it, who didn’t know or care and when? We don’t know this for sure and it is not the same thing is historicity but it would be expected to influence all the evidence we have.

    Other Points

    Should the probability of having the evidence we have be 1?

    The calculations in Carrier’s rebuttal of my video are correct. Disputes about mathematical modelling invariably turn not on the calculations themselves but the interface between the model and reality. So it is here. The two examples in play are my meningitis example and Carrier’s murder suspect. In both of these cases we can make a meaningful estimate of the prior probability. In the case of meningitis, it is the probability of having meningitis without knowledge of the test result. In the case of the murder suspect, we can formulate Bayes’ theorem as: the probability that the accused is guilty given that he has blood on him is equal to the probability of having blood on him given that he is guilty times the probability of being guilty all over the probability of having blood on him.

    The probability of having blood on him is a meaningful number that we can estimate. We can observe what proportion of people have blood on then and therefore decide how likely it is that that he would turn up with blood on him without reference to the rest of the evidence.

    Similarly in my meningitis example. The likelihood of having meningitis is something we can observe from epidemiological data and estimate without reference to the test result.

    In the case of Jesus, the probability of historicity given the evidence we have, is equal to the probability of the evidence we have given historicity times the probability of historicity all over the probability of having the evidence that we have.

    In this case the probability of having the evidence that we have is not something that we can meaningfully estimate. It is not analogous to the meningitis and blood examples.

    There are things that we can meaningfully estimate. These include the probability of having the evidence given historicity, also the probably of having the evidence given not historicity but add them together in you end up with the same problem.

    The estimate can be made meaningful but to do so it becomes trivial. We could construct a hypothetical situation with a number of different ancient histories and consider how many of them gave us the evidence we actually have. But as we don’t have these other ancient histories the matter gets us nowhere.

    Carrier’s misunderstanding is illustrated by him emblazoning this statement:

    The probability that the evidence exists given that we are observing it, and the probability that the evidence would exist given that a particular event happened in the past, are not the same probability.

    This is true, but neither of these probabilities are the probability at issue. What is at issue is the probability that the evidence would exist irrespective of what happened in the past.

    I assume we can agree that the probability that the evidence exists given that we are observing it is 1. But the probability that the evidence would exist irrespective of what happened in past is either 1 or not meaningful. Specifically what does it mean to say that the probability that: [the evidence would exist given historicity + the evidence would exist given not historicity] is less than 1? Or that the probability that the evidence would exist irrespective of what happened in past is less than 1?

    Carrier notes that if the denominator is one it defeats the purpose of using Bayes theorem at all. Yes, it does!

    How did I figure that?

    Very simple really. I converted Dr Carrier’s odds into probabilities for and against historicity for the four bodies of evidence and for the Rank Raglan hero class, and then simply took the average of the resulting 5 probabilities. So does this give a formal probability? No it doesn’t. And is it open to challenge? It certainly is. The method has a weakness that a compelling argument will be significantly diluted by trivial arguments. An improvement would be a points-based system where an argument favouring one side is allotted that side and given a weighting according to its strength. This is an attempt to model the decision-making process, as it occurs in scholarly discourse, with numbers. The reason for using it has nothing to do with the internal mechanics of the model, it has to do with the ubiquitous vulnerability of all mathematical models of the real world, and that is they may not describe the real processes accurately. My contention is that the geometric models so familiar from probability theory get the wrong result because of their failure at this interface, not because of any internal inconsistency.

    Weighing evidence

    Carrier is seduced by the elegance of probability theory and applies it without sufficient regard to whether his model fits the purpose. If you really want to develop a mathematical model to suit this question then what you need to do is to describe the actual process of decision-making by scholars in this field in mathematical language. That’s not going to lead to a model with the same elegance as theoretical probability but at least it will have a chance of describing the situation fairly. The way to do this is simply to weigh the evidence. There are multiple arguments in favour one side or the other. We need a method of assessing the strength of these arguments and comparing the two sides. The most obvious way of doing this is to collect all the arguments that favour of mythicism on one side and all those that favour historicity on the other. A necessary elaboration would be to account for the fact that strong arguments carry more weight than weak ones and therefore a point system springs to mind where strong arguments are allotted more points than weak ones. Then in the final analysis the points for historicity are added up as are the points for mythicism. Multiplying points is in no way analogous to the scholastic process.

    This is not the method I use which was simply to average the probabilities Carrier calculated from his five groups. I did this because it was the simplest way of arithmetically combining the probabilities that he estimated.

    Attractive theories are often shot down by inconvenient truths and here, as in many cases, the exponents of those theories recognise their limitations belatedly if at all. Review of previous discourse on this matter shows that I’m not the first person to point this weakness out to Carrier but he sticks to his position doggedly.

    Odds v probabilities

    Odds and odds ratios are extensively used in statistics and the more elaborate models are essentially geometric in nature and are not entirely convertible into arithmetic form. The Bayesian odds model Carrier uses can be expressed with exact equivalence as a probability model. His argument that I’m confusing to different things is fallacious. I chose to express his concepts with the probability version of Bayes’ theorem purely because I find this to be more accessible to people not accustomed to probability calculations.

    The version of Bayes’ theorem I expressed on screen is Bayes’ theorem. It can of course be made longer by expanding the various terms but it is in no way incomplete. There is nothing that this version hides from the lay observer. There is a word that describes the hidden subtleties that can only be appreciated by the initiate using an expanded version with extra variables and esoteric symbology and that word is bamboozle.

    We all know how this is going to go. Carrier is not going to back down. I know this for three reasons. If he understood the issue he would not have made the error in the first place. Secondly I’m not the first person to point this problem out to him and he has not in any way altered his position. Thirdly he has too much invested in it and can’t afford to concede the point. His acolytes will jump to his defence and we won’t see much discussion on where geometric models should and should not be used.

    But consider this. The next time you’re making a complex decision, with multiple input considerations, which method are you going to use? Carrier’s or mine?

    Fishers of Evidence

    Reply
    1. Holy crap. You were averaging the probabilities!? And when you did that you even just averaged the prior probability in with the likelihoods??

      I could drop the mic right there.

      I’ll let all the mathematicians of the world explain why you don’t know what you are doing.

      You also betray your ignorance of the fact that P(e) = [P(e|h x P(h)] + [P(e|~h x P(~h)]. You claim “the probability that the evidence would exist irrespective of what happened in past is either 1 or not meaningful” is simply not even remotely true. The probability that the evidence would exist irrespective of what happened is [P(e|h x P(h)] + [P(e|~h x P(~h)]. Which can never be 1, unless both P(e|h) and P(e~h) are 1. Which would entail there is no evidence for the existence of Jesus. Exactly as I explain in my article. That you don’t know that, proves you don’t know Bayes’ Theorem.

      As to the rest:

      I actually give data-based reasons for all my estimates of probability. I account for the uncertainties in the data, and possible biases and subjectivities, with margins of error so wide it would be unreasonable to believe the probabilities deviate further. If you wish to gainsay that, you have to actually make a case. You can’t just insist the probabilities are different, and then simultaneously insist no one can know what the probabilities are. If you can never show how they could be different, you can’t claim to know we aren’t using reasonable margins.

      And even if we granted no estimates of probability were defensible in this case, you then can’t claim to know the historicity of Jesus is probable—if you say no one can assign and calculate any probabilities of anything pertaining to answering that question, then you are saying no one can know if Jesus existed.

      So the only way to get a different result than “historicity agnosticism” (which means fully doubting the existence of Jesus) is to say what you think these probabilities are, and explain why that’s what you think they are, and not something else. You can’t say “no one can know what they are, therefore Jesus probably existed.” That’s full on illogical.

      So either you are just arguing for agnosticism about the existence of Jesus, by saying no one can ever know anything about the past (or at least the ancient past of human events). Or you are trying to argue for a different probability than I am, without giving a single argument for a single instance of that being true, while simultaneously (and illogically) insisting you can’t know that. I have given reasons in OHJ. For every single probability I assign. Good reasons. And indeed, good reasons for doubting it’s reasonable to put the probabilities any further out than I do. You are refusing to address any of those reasons. Nor, in any single case, are you presenting any comparable reasons for putting the probabilities any further out than I do!

      You aren’t discussing the actual probabilities of anything or why they should be different. You are illogically trying to get a probability out of multiplying arbitrarily invented “points” (??), and then you claim using actual probability mathematics is less valid, and more arbitrary? And then you are proposing a mathematically incorrect procedure of averaging the probabilities of individual events to determine the total probability of a conjunction of those events! And averaging the prior in as if it were another event in the conjunction! And then claiming P(e) is 1, which entails no evidence exists for Jesus!

      It’s clear you have no idea what you are doing. It’s also clear by your own assertions that you do not know, and indeed are declaring you cannot know, whether Jesus probably existed.

      Pick a lane.

      Then learn how to actually do the math.

      Then try actually critiquing the actual evidence and arguments presented in OHJ for the assigned probabilities.

      Anything else is a waste of everyone’s time.

    2. Fishers of Evidence wrote:

      “Carrier’s probability range of 0.12% is also suspiciously low.”

      You give absolutely no reasons for this claim that are related to the case at hand. Your only argument is that the number is “low”. If that was a valid argument, you would have to agree that *any* low probability outside of theoretical models is suspicious. For instance, if someone says that the probability that your mother was a hampster is 0.12% you would have to complain that surely we cannot say that it is that low. But we can!

      You may have a “hunch” that it is low, because you believe you have relevant knowledge that makes this seem improbable. But then you have to examine your knowledge and motivate why it should be higher.

      “[..] the estimates will vary from one person to another and that means that they are arbitrary. That is what the word means. If a staunch historicist addressed the same data, it is possible that they would revise their views and become a mythicist but it is also possible that they would come up with different probabilities.”

      This is correct, but this is not a flaw in the method, quite the opposite! If a serious historicist actually did this, we would first of all get a combined range for what serious scholars consider plausible. Perhaps that range would be very large, perhaps it would be small enough that we can say they are pretty much in agreement. And if we get a large range, we will see which arguments are the main causes of this disagreement. Those would be the areas where further research should be focused. That is how we make progress in science, by focusing on areas of disagreement and examining the facts until most reasonable scholars are more or less in agreement.

      “Specifically what does it mean to say that the probability that: [the evidence would exist given historicity + the evidence would exist given not historicity] is less than 1?”

      It means that it is not certain that the evidence would exist. For instance, let’s take a spam detector. The probability that a spam email contains the word “viagra” is much lower than 1. The probability that a legitimate email contains that word is even lower. The probability that an email, whether it is spam or not, contains the word “viagra” is much lower than 1. Spam detectors often use Bayes’ theorem to calculate the probability that an email is spam given many indications, where the occurrence of the word “viagra” is one factor. And yes, they will, of course, multiply these probabilities. Even though they are not necessarily independent.

      “The method has a weakness that a compelling argument will be significantly diluted by trivial arguments.”

      Indeed. Using that method, we can add irrelevant arguments until the probability of *anything* becomes 50/50. Vaccines are poisoning the children? Lizards are controlling the world? It would all be 50/50. That’s obviously completely useless, which is why you will not find this method in any textbooks. Really, you should be more careful about thinking that you can invent something that is superior to what all the mathematicians have been able to do.

      There is, I think, a risk involved with combining subjectively judged probabilities. This risk is that if the probability of any single argument is judged to be very low, it can easily overcome arguments pointing in the other direction. To avoid getting mislead by this, we should be careful of assigning very low probabilities, especially for hypotheses we are arguing against. You will notice that Carrier has not assigned any such very low probabilities. Instead, he ends up with a low combined probability because he believes there are many facts that are at least moderately improbable on historicity.

Add a Comment

I only publish comments by my patrons and anyone who or whose work I discuss in the article commented on. Comments must also follow good etiquette. Those who engage in dishonest, abusive, or harassing behavior may even be banned as commenters and patrons.

 

*