you can register now!Proving HistoryAviezer TuckerFebruary 2016 issueHistory and Theoryabstract of his reviewsolveProving HistoryProving HistoryOn the Historicity of JesusEveryone Is a Bayesian
Bayesian Reasoning in a Nutshell
- Rule 1: Estimate probabilities as far against your assumptions as you can reasonably believe them to be.
Proving Historya fortiorican’treasonablya lot lessa fortiori
- Rule 2: Estimate the prior probability of the conclusion you are testing.
usuallyusuallyelseotherprinciple of indifferenceProving History
- Rule 3: Prior probabilities are always relative probabilities. Because the prior is an estimate of the frequency of a claimed cause of the evidence relative to all other things that could have caused the same evidence.
relativetosomethingnotthatrelative to alternatives
- Rule 4: Estimate the probability (also known as the “likelihood”) of all the evidence as a whole if the claim you are testing is true.
assumeallProving Historyyou
- Rule 5: Estimate the probability (also known as the “likelihood”) of all the same evidence if the claim you are testing is false. Which always means: if some other explanation is true.
you have tohigherlikelihoodhassomethingothersymptomsotherbest
- Rule 6: The ratio between those likelihoods (generated by following rules 4 and 5) is how strongly the evidence supports the claim you are testing. This is called the likelihood ratio.
just as expectedalmost as expectedmanynot very expected at allbutextremely improbablisn’tthe ratio
- The odds on a Claim Being True = The Prior Odds times the Likelihood Ratio
probabilitiesother
Prior Odds [x] Likelihood Ratio = 1/100 x 100/1 = 100/100 = 1/1
thatthousand
Prior Odds [x] Likelihood Ratio = 1/100 x 1000/1 = 1000/100 = 10/1
I’ve Said It Before
Guest is first bothered by not knowing where I get my estimates from [in a historical analysis]. But … they are just measures of what I mean by “unlikely,” “very unlikely,” and similar judgments. My argument is that “assigning higher likelihoods to any of these would be defying all objective reason,” … which is a challenge to anyone who would provide an objective reason to believe them more likely. In other words, when historians ask how much [a certain piece of] evidence weighs [for or against a conclusion], they have to do something like this. And whether they do it using cheat words like “it’s very unlikely that” or numbers that can be more astutely questioned makes no difference. The cheats just conceal the numbers anyway (e.g., no one says “it’s very unlikely that” and means the odds are 1:1). So an honest historian should pop the hood and let you see what she means.
Math Doesn’t Suckwhat you meanUnderstanding Bayesian History
Historians are testing two competing hypotheses: that a claim is true vs. the claim is fabricated (or in error etc.), but to a historian that means the actual hypotheses being tested are “the event happened vs. a mistake/fabrication happened,” which gives us the causal model “the claim exists because the event happened vs. the claim exists because a mistake/fabrication happened.” In this model, b contains the background evidence relating to context (who is making this claim, where, to what end, what kind of claim is it, etc.), which gives us a reference class that gives us a ratio of how often such claims typically turn out to be true, vs. fabricated (etc.), which historians can better estimate because they’ve been dealing with this kind of data for years. We can then introduce additional indicators that distinguish this claim from those others, to update our priors. And we can do that anywhere in the chain of indicators. So you can start with a really general reference class, or a really narrow one—and which you should prefer depends on the best data you have for building a prior, which historians rarely have any control over, so they need more flexibility in deciding that (I discuss this extensively in chapter 6 [of Proving History], pp. 229-56).
what caused the evidence we now havesecond all the forty or so GospelsOn the Historicity of Jesusnot knowing whether it isn’tambiguity intolerancemore probably
Concluding Observations for the Future
If You Learn Nothing Else about Bayes’ Theorem, Let It Be Thissomething else is true insteadprior probabilityPrior ProbabilityThe Odds, Continually Updated
Now Bayesian statistics are rippling through everything from physics to cancer research, ecology to psychology. Enthusiasts say they are allowing scientists to solve problems that would have been considered impossible just 20 years ago. And lately, they have been thrust into an intense debate over the reliability of research results.
But you can’t do that and claim to be making valid inferences
Bayesian calculations go straight for the probability of the hypothesis, factoring in not just the data from the coin-toss experiment but any other relevant information—including whether you’ve previously seen your friend use a weighted coin.
in generalif you knowmultiple comparisons fallacythat this is happening in the field of psychologyfor ages nowAlex Birkettpwned the entire male mathematics establishmentwhich door he didn’t openprior informationPrior ProbabilityProving HistoryBayes’ Theorem: Lust for Gloryto the Tim Hendrix reviewTwo Bayesian Fallacies
On the Marilyn vos Savant incident with the Monty Hall problem. It’s true that many mostly male mathematicians fumbled this; but Marilyn herself was not blameless.
Her presentation of the problem was incomplete. I do not recall the details, but she was not clear in her exposition that in the problem, Monty *always* opens a door that *does not* contain the prize. This is absolutely necessary for the puzzle to work.
For example, if Monty were always to *randomly* pick a door and open it, and it *happened* not to have the prize on this particular occasion (I call this version “Ignorant Monty”), then the probability does revert to 1/2 and it doesn’t matter if you switch doors.
Or, if Monty opens the door with the prize and says “Ha, ha, you lose” every time you pick the wrong door, but on this occasion you’ve picked the right door and he offers you the chance to switch (I call this version “Monty From Hell”) then you should *never* switch, since Monty From Hell’s opening a door is proof that you have picked the prize door.
Or, Monty could open your door if you have guessed right and say, “Congratulations, you’ve won!”, and only offer you a chance to switch if you’ve chosen wrong. I call this version “Angelic Monty”, and obviously if that’s the version being played, you should switch if he offers you the chance because then you are sure to win (the door he opens and your original door don’t have the prize, so the only one left does).
I challenge my students to tell me what they should do if they are offered “Mixture Monty”. In this version, Monty flips a coin backstage and if it comes up heads he behaves like “Angelic Monty”, and if it comes up tails he behaves like “Monty From Hell”. Assuming the coin is fair, should you switch if offered the chance? What’s the probability of winning if you do?
So, as I said, Marilyn wasn’t blameless in this incident as she was not clear in her presentation of the problem.
This problem was discussed in a professional journal (American Statistician, as I recall), and Monty himself weighed in, pointing out that he did not in fact use the rules of the standard mathematical puzzle, but behaved more like “Mixture Monty” in that he only offered a chance to switch sometimes, and not always when the contestant had chosen rightly (or wrongly). It was, therefore, just an additional occasional feature of the game.
No, you’ve slightly misremembered. The article I linked to covers all this.
Hall said what he did differently was bribe the contestants, and her analysis only skipped that part (since, being psychology, it wasn’t interesting to her as a mathematician; and indeed, no version of the Monty Hall problem since ever incorporates the role of bribes).
And Vos Savant was very clear on the rules of the game. Her letter is here. It’s of course obvious anyway, since it’s the only way for the game to work. But nevertheless she is explicit, that Hall only opens doors that don’t have the prize. She even used the million-doors example to show this.
I was referring to the *original Parade article*, which I did not misremember and which did *not* clearly state that Monty *always* opens a door that does *not* contain the prize.
In fact, I wrote to her pointing this out.
In her subsequent discussions, such as the one you linked to (which was not the original article), she was clearer in her statement of the problem. I can hope that what I sent to her after her original Parade article encouraged her to write more clearly.
Oh. I was under the impression the link contains all the original material. Do you have scan of the original?
Let me be a little more clear.
If you look at the link you gave, the original problem was posed by Craig Whittaker. If you read the problem as he stated it, you will see that in no way does it say that Monty *always* opens a door that does *not* have the prize. All it says is that he opened a door (on this particular occasion) and that it did not have a prize (on this particular occasion).
The problem as stated by Mr. Whittaker could have produced the evidence observed, regardless of which of the several different kinds of Montys actually pertained to the problem. That’s because, according to the statement of the problem by Mr. Whittaker, the contestant only knows (a) that Monty knows where the prize is, (b) that Monty has opened a door that is not the one you chose and (c) that the prize was not behind the door he opened. *Any* of the Montys I described could have produced this evidence for this particular contestant. And Marilyn made no effort to correct the problem *as stated by Mr. Whittaker* to bring it into the canonical form.
Marilyn does not correct this, though I agree that her answer *assumes* the standard puzzle. But this is my complaint. She should have stated the complete problem including all side conditions (what we Bayesians know as background conditions, stuff on the right side of the conditioning bar). That was my complaint about her discussion of this problem.
But Vos Savant does correct him, by explaining that of course he doesn’t throw the game, and that he only ever opens what is empty, how the game actually works. (And Monty Hall later corroborated her.) So the actual game Mony plays is the game she analyzes. As she ought to have. And she did make this clear. No one who challenged her said she was wrong about that.
Unless…
You’re right that the link you gave does include both the original question and Marilyn’s answer, as published in Parade. Sorry to miss that fact.
However, this supports what I am saying. Here’s what was originally published. First, Craig Whittaker’s question:
—-
Suppose you’re on a game show, and you’re given the choice of three doors. Behind one door is a car, behind the others, goats. You pick a door, say #1, and the host, who knows what’s behind the doors, opens another door, say #3, which has a goat. He says to you, “Do you want to pick door #2?” Is it to your advantage to switch your choice of doors?
Craig F. Whitaker
Columbia, Maryland
—-
Then, Marilyn’s answer (the answer as given in the original column):
—-
Yes; you should switch. The first door has a 1/3 chance of winning, but the second door has a 2/3 chance. Here’s a good way to visualize what happened. Suppose there are a million doors, and you pick door #1. Then the host, who knows what’s behind the doors and will always avoid the one with the prize, opens them all except door #777,777. You’d switch to that door pretty fast, wouldn’t you?
—-
That’s all. Everything else in your link is stuff that was published *after* the original comment and *after* I also wrote to Marilyn to complain that she hadn’t been clear about the conditions under which her answer is correct (e.g., that Monty *always* makes this offer to *every* contestant after open a door that he *knows* does not contain the prize). Alternatively, (and more generally) it needs to be clear that Monty’s decision to make this offer is *independent* of the contestant’s choice of door:
P(Monty offers switch|Contestant chooses some particular door)=P(Monty offers switch).
For example, Monty could decide to offer the contestant the switch only if the contestant had chosen right (he didn’t do that, but that’s immaterial to the point I am making). If he did that, and just didn’t offer the switch when the contestant chose the wrong door, then it would *never* be a good idea to switch (but his decision to offer the switch would no longer be independent of the door the contestant chose, violating the condition above).
This is even true in the million door example. If you happened to choose the correct door, and Monty only offered the switch in this (unlikely) case, you’d still lose for sure by switching.
There is nothing that I can see in either Mr. Whittaker’s statement of the problem or in Marilyn’s answer *to him* as given above that makes any of this clear. If I am mistaken, please point out where her answer *above* makes it clear that Monty makes his decision to offer the switch independently of the door the contestant chose.
You’ve commented that she makes it clear “by explaining that of course hd [sic] doesn’t throw the game, and that he only ever opens what is empty,” but any comments like that are *not* in her original answer above, which is the only thing I am commenting on since it is her original comment that led all of those mathematicians to say that she was wrong (incorrectly…if you use the canonical form of the puzzle as given originally, I believe, by Martin Gardner).
In other words, your comments that rely on later comments by Marilyn are not relevant to my original complaint…I could hardly have read them before she published her second column taking the many mathematicians to task and where she was more clear in her exposition…since I had long since written her to complain about her unclear answer in the first place!
You’ve noted additionally that Monty himself made some comments, but they aren’t in the link you gave, and I haven’t been able to locate the exchange on this topic that appeared in *American Statistician*, though I’m pretty that’s where it was. Do you have a link to Monty’s comments?
Even with that briefer version, when she said “who knows what’s behind the doors and will always avoid the one with the prize” she is correctly describing the scenario, and her math follows from what she describes. Which is what people should have attended to. They could have challenged her by saying “Yes, you are right in that case, but if he opens boxes randomly, and thus doesn’t prevent the prize being revealed when he offers to open one, then the result is different.” Did anyone correctly say that? Or did they all ignore the line “who knows what’s behind the doors and will always avoid the one with the prize”? 🙂
“Even with that briefer version, when she said “who knows what’s behind the doors and will always avoid the one with the prize” she is correctly describing the scenario, and her math follows from what she describes. ”
Yes, that’s what she wrote. And it correctly describes the scenario on any occasion where Monty DECIDES to offer the contestant a switch.
But “her math follows” ONLY if Monty chooses to offer the switch INDEPENDENTLY of which door the contestant has chosen.
THAT is the whole point of my complaint about her response to Whittaker’s question.
Where does her comment say, directly or indirectly, that Monty will DECIDE to offer this choice to any contestant INDEPENDENTLY of which door the contestant has chosen? Or, alternatively, that Monty ALWAYS DECIDES to offer this choice?
As I said, if Monty had a rule (unknown to the contestant) that he would only offer the switch if the contestant chose the right door, or even if Monty had a rule that he would only offer the switch if the contestant chose the right door *sometimes* (with some probability that would keep the game going but would not be independent of what the contestant had chosen), then the “standard answer” to the Monty Hall problem will be wrong. It would be badly wrong if Monty *never* offers a switch.
Chapter and verse, please.
I’m not sure I follow you. The correspondent said “the host, who knows what’s behind the doors, opens another door.” There are only three doors. Monty offers to open one of the doors the contestant didn’t pick. That’s plainly stated in the problem, the problem she was asked to solve. She was asked should you switch, after that has happened. So whether he always does this is not relevant to what she was asked.
Or am I misunderstanding something you mean?
Oh wait, I think I see what you mean: that if Monty only ever does this when the prize is behind the contestant’s door, then it changes the probability to zero for switching.
I think that’s why no one would ever think this. Because then everyone would know as soon as Monty offered to open another door, he was telling them they already chose correctly. That would be foolish. I can see how it remains logically possible, just not why anyone would ever think Monty would engage in such a self-defeating pattern. But yes, one could add that as a condition that would change outcomes (you would then need to know another variable: the frequencies with which Monty does this when the contestant’s box is correct and when it is not; which not being stated in the problem posed, are unknowns).
Right, you’ve got it in your last comment, Richard.
You are also right that if Monty *only* does this when the contestant has chosen the right door, then (if contestants have memory of the show) after a while no contestant will take the bait.
This is why I posed the challenge to my students, of a “mixed” strategy: Sometimes Monty behaves like “Angelic Monty” (offers the switch if the contestant chooses wrong) and sometimes behaves like “Monty From Hell” (offers the switch if the contestant chooses right). He could (and in my problem to my students, does) decide which strategy he’s going to employ, in advance of the contestant’s going on stage…it’s predetermined. He would do this by some chance operation (in the problem to my students, I say he flips a coin so it’s 50/50 Angelic/Hell, but it could be any random process with any odds, just as long as the decision is made in advance of the game).
In the situation of the problem I pose, Monty always opens a door, and if the door he opens is empty, the contestant won’t know if he’s facing Angelic Monty and has picked the wrong door, or Monty From Hell and has picked the right door. Again, nothing in either Whittaker’s posing of the problem or Marilyn’s reply (applied to the 3-door version) will inform the contestant as to whether the game being played *this time* is Angelic/Hell or the standard Monty as described by Martin Gardner, since in this particular case an empty door has been opened. Nothing in Whittaker’s question or Marilyn’s reply says anything about multiple games, it is only about the particular game played with this particular contestant.
My argument with Barnes hinges on the necessity to take into account prior information that you may have, since as Jaynes points out, failure to take such information into account often leads to nonsense answers, as it does in the “fine tuning” arguments for the existence of a deity.
This happens because it is common for people to leave out background information when writing out Bayes’ theorem. Usually this does not pose a problem since everyone “knows” about the background information, and obviously much background information that we have will be irrelevant to the problem (i.e., the results of Bayes’ theorem do not depend on that information being specified since the data are independent of that information…the color of the socks I am wearing should not affect my inferences about whether the Sun will rise tomorrow! But the existence of the observer *does* affect inferences about fine tuning.)
In the Monty Hall problem, it’s my point that one needs to specify as background information the rules that Monty is following in the game. The contestant only knows what’s happened in the particular case at hand…but the rules that Monty follows, whether he opens a door at all, for example, the probability that he opens a door that shows the prize or that does not show the prize, matters for the probabilities and therefore whether the contestant “should” switch. There’s an unstated bit of background information that’s needed to get the “standard” result that the prize is behind the remaining unopened door with 2:1 odds, and that was my original complaint to Marilyn. Unlike all the mathematicians, I knew that the answer she gave was “correct” for the *standard* problem, but I faulted her for not making the conditions of the *standard* problem clear. Had the problem been a different one, e.g., Monty From Hell, Angelic Monty, Mixture Monty, or Ignorant Monty (where he decides in advance to open a random door and it happens not to have the prize), then the answer would be different, given the same information available to the contestant (I chose a door, Monty opened another door that didn’t have the prize, should I switch?)
So: Suppose the contestant is facing Mixture Monty, with the odds of Angelic/Hell decided by the flip of a fair coin. Richard, should he switch if he knows he is facing this particular Mixture Monty?
Also, Richard, can you post a link to Monty Hall’s comments, which I haven’t been able to locate?
Fascinating. Right, if we complicate the model with all that, we have a much bigger problem.
On Monty’s comments, I thought those were in the article I cited at the start? If not, I can’t recall then where I read them.
I found the article you cited containing a comment by Monty. I was looking at the wrong link. Here it is:
http://priceonomics.com/the-time-everyone-corrected-the-worlds-smartest/
I still can’t find the letter in American Statistican! If I do find it I’ll let you know.
Thanks, Richard!
No, that I don’t know the exact reference for.
Whats the deal with Ralph Ellis?
Explain what you mean. I have no idea what you are talking about.
He has theories about Jesus.
Oh. I’m uninterested in random theories about Jesus. There are hundreds of those.
Hiya Richard,
My count of people hallucinating Jesus alive is 514 from Paul. I don’t have reason to think they were all liars and it is background that hallucinating gods happened a lot in that era. Doesn’t that make the odds of some Jesus believer hallucinating Jesus alive one hundred per cent because we know that is what actually happened, and more than once? I am not Everyman but after I understood to read Paul knowing the Gospels were a generation later at least, that they were ad-hoccery couldn’t be un-made. A stolen corpse is a more probable solution to the Gospel resurrection story, sure, but reading Paul in context blows the Gospels right out the water. I doubt if someone is not prepared to get that (and hardly any are; just look at the numbers of even atheists reading the Gospel Jesus back into Paul) that arguing for a stolen corpse being more likely than a resurrection is going to have any impact on their beliefs.
Shivam, “Whats the deal with Ralph Ellis?”. Based on looking at his website for ten seconds, he’s a loony. Why are you asking incomplete, daft questions? If we are going to talk probabilities, I think the odds of the most extravagantly daft Jesus theory you or I could come up with having already been proposed by some loon on the internet are approaching 100%. The same that, if not the Monty Hall problem, some other part of this post will result in someone arguing the toss for pages and pages of comments and the maths getting more and more arcane. ‘Math Doesn’t Suck’ might get us all up to speed but I keep seeing plenty of folk educated beyond sixth grade arguing the maths and lining up authorities for those arguments. Seriously guys, if you have to go beyond a couple of comments or paragraphs, take it up with Richard in PM. It gets really old really fast.
[P.S. ‘I discuss this method in detail in Proving History (index, “a foriori, method of”).’ You’ve dropped the t in “a fortiori”. — Thanks, fixed, RC]
Thousands simultaneously hallucinate visions of the Virgin Mary. And when we look at the only record of a large hallucination event in Paul’s timeline (the only record of over a hundred “seeing Jesus” after his death), it’s Acts 2. In which what is happening is just like a stylized version of Marian apparitions, where it’s just a vague visual hallucination and a feeling.
So there’s no difficulty there.
But I doubt Paul wrote quite what we have in 1 Cor. 15:6 anyway, “then he appeared to above 500 brethren at once.” Acts says it was 120 (Acts 1:15), which is a stylized biblical number. The author of Acts used the letters of Paul. So if that author had seen “over 500” in verse 6 of Paul, they would have used that number in Acts, as it’s more impressive than 120. So this convinces me the “over 500” was not in the letter of Paul at that time. In fact the Greek spelling is just a few letters away from a completely different sentence, and scribal errors of a few letters are very common in textual transmission. With an easy mistake, the sentence could have become what it is now, from an original “then he appeared to all the brethren on the Pentecost at once,” matching Acts 2:1, that all the brethren shared a vision of the spirit of Jesus on “the day of the Pentecost, when they were all together in one place.” This would explain why Luke placed this event on the Pentecost (because it’s where Paul originally did; the words for “Pentecost” and “Five Hundred” are nearly identical), and why Paul would refer to Jesus as “the firstfruits of the resurrection” (1 Cor. 15:20, a reference to the Pentecost).
For details see my analysis (and scholars supporting some elements) in The Empty Tomb, p. 192, and n. 371, p. 231.
Note that this is the only event Paul says was simultaneous. That he specifically says only here “all at once” means none of the other appearances he lists were group appearances (so, just as verse 7 does not mean “James and all the apostles” at one occasion but a sum of individual appearances, so in verse 5 he does not mean “the twelve” saw Jesus on one occasion, but each of the twelve had a separate individual experience…or claimed to have; we should remember, not everyone need be honest).
In Mark, the cry of dereliction from the cross, as well as the prayer in the Garden Of Gethsemane, seem to speak against the interpretation (usually from Catholics) that Jesus was one third of the trinity. In these two cases, Jesus was clearly not crying out to himself (which would be absurd,) but to another: to God.
The cry of dereliction in Mark also shows Mark’s Jesus did not believe he would be resurrected after three days, because there is no reason for Jesus to make the cry of dereliction if he is expecting a speedy resurrection.
Quite the opposite. That cry fits Jesus to a standard Jewish trope of the abandoned then vindicated righteous man. This is well explained by John Meier (see for discussion and scholarship: Proving History, pp. 131-33).
Correct. The trinity was invented after the Synoptic Gospels were written. For what Christians originally taught about that, see OHJ, pp. 92-96 (Element 10).
William Briggs, a professor of statistics who works at Cornell, has published a critique of your use of Bayes’ Theorem here: http://wmbriggs.com/post/13390/ . Though this has more to do with your use of it in the context of fine-tuning arguments, he also makes general challenges to your methodology. I am curious to know how you would respond to two of them, namely his claims that [1] while Bayes Theorem is useful in “updating” probabilities, it is not necessary for assessing them at the outset, and [2] that probability “measures information,” which can be but is not always in the form of frequencies.
For convenience I include the relevant quotes. In the section of his paper titled “Bayes,” he says: [1]. “Bayes’s theorem is a simple means to update the probability of a hypothesis when considering new information. If the information comes all at once, the theorem isn’t especially needed, because there is no updating to be done. Nowhere does Carrier actually needs [sic] Bayes and, anyway, probabilistic arguments are never as convincing as definitive proof, which is what we seek when asking whether God exists.” He supports this with an example from card playing. And later he states [2]. “Carrier further shows he misunderstands his subject when he says ‘Probability measures frequency’. This is false: probability measures information, though information is sometimes in the form of frequencies, as in our card example. Suppose our proposition is ‘Just two-thirds of Martians wear hats, and George is a Martian.’ Given that specific evidence, the probability ‘George wears a hat’ is 2/3, but there can be no frequency because, of course, there are no hat-wearing Martians.'”
I am no expert here, but these statements seem plausible to me and they come from someone with relevant credentials, so I would be happy to see your responses.
On those general questions, “[1] while Bayes Theorem is useful in “updating” probabilities, it is not necessary for assessing them at the outset” is not something a Bayesian would ever say, so it sounds like he has an axe to grind against Bayesians—obviously you can’t update a probability if you don’t start with one! Maybe you have to pick one by estimate rather than a previous run of Bayesian equation, but that’s exactly what I’m talking about doing. Although in reality that estimate is just the estimate of a previous intuitive run of the equation (e.g. you are estimating what the result would have been if you worked the Bayesian numbers all the way from raw data—which means all the way from properly basic uninterpreted sense experience). Of course, if you don’t know anything about what the starting probability is, then you can’t make a fine tuning argument in the first place. So the probability has to come from somewhere. And this is what I discuss when I treat questions like the threshold probability (hence read up here). Which is not derived by Bayes’ Theorem but from the data, which include information about the possibility space we are trying to run a probability distribution on.
As to “[2] that probability “measures information,” which can be but is not always in the form of frequencies,” that’s an assertion with no evidence to back it. I guarantee, anything he says about information from which he derives a probability, will always translate to a frequency of something. That consequence can’t be avoided. I think again he is confusing one kind of frequency statement for another, e.g. he is ignoring the fact that stating degrees of belief is making a statement about the frequency of being wrong.
I should add that Briggs seems to think this is my argument. It’s the argument of three expert mathematicians published separately in two sources. Which Briggs ignores and does not engage with at all.
When I look over the Briggs article, he’s just blathering with insults and misrepresentations of their argument (as if it were my argument). It’s hard to take him seriously. He doesn’t actually engage with my argument as presented in TEC either. And his arguments make no sense. For instance, saying that “2/3rd of Martians wear hats” is not a frequency because Martians don’t exist is to confuse the meaning of a statement with its referent. This is a boner mistake for someone who is supposed to be a professor. Obviously saying that “2/3rd of Martians wear hats” is stating a frequency…about a hypothetical set; that’s what the sentence means. Whereas if he thinks this is a relevant analogy, he’s saying fine tuning arguments are impossible because we can’t make frequency statements about hypothetical sets, and yet fine tuning arguments require doing that, e.g. making statements about the frequency of biophilic universes in the set of all nonexistent universes (just like nonexistent Martians).
I can see no merit to anything he’s arguing. It really is a bunch of illogical crap like that. And it doesn’t engage with my actual argument at any point (much less the arguments of Ikeda, Jefferys, or Sober).
I say just read my article in TEC. You can see yourself how he isn’t actually making any valid objections to it.
I agree with your assessment of Briggs’ comments. He’s also got some ideas about physics that are incorrect.
Luke Barnes and I do not agree about his “fine-tuning” arguments, which I believe to be fatally flawed. He doesn’t like my arguments against his arguments (see my article with Michael Ikeda, in The Improbability of God, edited by Michael Martin and Ricki Monnier, Prometheus 2006).
That’s a good book all around, too.
To be clear, I’m agreeing with Richard Carrier about Briggs’ comments.
I have no idea if he’s right, but when it comes to Bayes, Luke Barnes has pretty strongly claimed you don’t know what you’re talking about, here:
https://letterstonature.wordpress.com/2016/02/01/richard-carrier-one-liners-part-3/
He always does. I’ll look over what he means this time though.
Response here.
Hi,
I am a longtime reader and am really interested in your novel approach to the study of history. I have also considered taking your course on Bayesian reasoning. Yet I have been reading some pretty powerful critiques of your work and I am starting to have doubts about your competence in statistical reasoning.
I read your post a couple of weeks ago that criticized Kyle Barnes, and I also read the comments. In the comments when you were talking with Tim Hendrix it appeared that you wetter confused about some concepts.
I noticed today that Barnes has a response to you posted on his blog:
https://letterstonature.wordpress.com/2016/01/29/richard-carrier-one-liners-part-1/
https://letterstonature.wordpress.com/2016/01/31/richard-carrier-one-liners-part-2/
https://letterstonature.wordpress.com/2016/02/01/richard-carrier-one-liners-part-3/
These critiques of you are pretty damaging to your credibility. It again appears you have misunderstood basic concepts of Bayesian statistics. Barnes also links to a blog post where another person posted the texts of one of your courses on Bayesian reasoning and proceeded to point out scores of errors.
Do you have a response too any of this? I really am a longtime fan, and this is all very disappointing to me.
I’ll take a look.
I’ve responded here.
The McGrews are probably not worth responding to. They are haphazardly commenting on an obsolete six year old document I never use anymore. And their comments on it are mostly trash talk. That old document is pedagogically a nightmare of wording anyway. My book Proving History completely replaces it.
I have a question about how you ascribe uncertainties to calculations using Bayes’ Theorem. Let’s use a trivial example from the Wiki page:
My question is, suppose the inputs are given with uncertainties, e.g.
P(A1) = 0.2 ± 0.1, P(A2) = 0.3± 0.1, P(A3) = 0.5± 0.05
That is, there is some uncertainty about the relative fractional outputs. And also in the defect rates, viz.
P(B | A1) = 0.05± 0.01, P(B | A2) = 0.03± 0.01, P(B | A3) = 0.01± 0.005.
What would then be the uncertainty in the final answer of 5/24?
I didn’t check that math to see what they mean (whether 5/24 is a probability or the odds).
It sounds like you are asking how you convert odds to probabilities. That’s a procedural question. When you are asking about total odds, you sum the denominator and numerator and divide. So, 5/24 translates to a probability via 5/29 = 17% (approx.). (I didn’t check the Wiki’s math; I’m just assuming they correctly derived the ratio as 5/24 and that they mean this as the total odds, which might not be the case.)
But rather than burn time on this elaborate example, there is a simpler answer to your question, and it’s precisely what I do in OHJ (see Chapter 12.1): run the math for the odds all at the margin in favor, then run the math for the odds all at the margin against; the two results will be your range of uncertainty: from best odds to worst (within your chosen margin of “reasonable certainty”).
Alternatively, don’t use odds or fractions. Just stick with probabilities. You can do that even with the Odds Form of BT, but it requires you to know the whole formula (see Proving History, Appendix), since you then are plugging in probabilities to get the odds. That full formula is: P(h/e.b)/P(~h|e.b) = P(h/b)/P(~h|b) x P(e/h.b)/P(e|~h.b).
Hi Bill,
This is a really good question. What you *should* do of course depend on various things and I would rather not turn this into a long argument, however you should be aware that what Richard is proposing is very different from what a Bayesian statistician would do. What any Bayesian statistician would do is as follows:
You would first describe all your probabilistic statements as probabilities, for instance you would introduce a variable x such that
P(A1) = x
and then that x followed some distribution (typically a Beta distribution since they take values between 0 and 1). Then when you did your computation, you would apply the only tool you have available: the rules of probability theory and compute (for instance, and I am sorry this is looking so awful)
p(B) = int [ p(B|A1,x)p(A1|x) + p(B|not A1,x)p(not A1|x) ]p(x)dx
= int [ p(B|A1)x+ p(B|not A1)(1-x)]p(x)dx
= p(B|A1)Ex + p(B|not A1)(1-Ex)
where Ex is the mean of X under its respective prior distribution p(x). So a Bayesian approach to probabilities gives a self-contained approach to uncertainty in probabilities :-).
Regarding Richards proposal, the reason why this is not used is
1) that a statement such as 0.1+- 0.2 is likely a statement about a frequentistic confidence interval or some such which is avoided on a Bayesian account of probabilities (i.e. it should be translated as a statement about probabilities if this is information we have available)
2) if the variable is 0.1 +- 0.2; normally we mean it could also be 0.9 (but with very low probability) and if these extended ranges are used our min/max statements are going to be overly wide
3) if we take the min/max our result is not representative of the typical behavior of the model
4) its just ad-hoc with no theoretical foundation.
Indeed.
Also note, that this question is about my use of margins of error, which I have only done in history, not the fine tuning argument. That context needs to be reintroduced:
1) is also what I said: if you are going to use percentages as error margins, you have to run the whole thing with percentages and not odds. But that gets complicated for humanities majors, which is why I recommend simpler methods; we don’t usually need the precision of more technical methods there, because we don’t have the precise data that warrants it. It would be like using a three stage rocket to commute to work.
2) this is exactly what I recommend in PH (and run in parallel to the odds method in OHJ). The concern that our min/max is going to be too wide is a category fallacy. It is simply the case that in history, our margins of error often are that wide. And we have to be honest about that. As I explain in PH, that’s what makes the difference between history and science: we often do not have scientific certainties.
3) historians aren’t asking about the typical behavior of a model, they are asking whether a claim is true or false. There are only two options there.
4) I don’t fathom what’s ad hoc about calculating the a fortiori posterior and calculating the a judicatiori posterior. The full theoretical foundation for this comes from the philosophy of history and is detailed in PH. Quite simply, this is necessarily what you must do when the two questions you are asking are, “what is the highest probability of being true that I can reasonably believe this claim has?” and “what is the lowest probability of being true that I can reasonably believe this claim has?” That’s simply what historians are doing. And I am modeling what historians do. Scientists can ask different questions and do different things.
Yes, he always does, and you respond by calling him insane or a kook. The debate has devolved to name-calling on both sides, but I suppose that’s just what happens when two people are both sure they’re correct. Better if we could do without that, but that may be more than we can expect from mere mortals.
Regardless, there are some things I think a lot of us are hoping to see explained, and since Barnes has been exhaustive (and prompt) in responding to your commentary, so far as I can tell, one hopes you can be similarly responsive to his posts, including his questions addressed to you:
1. What are the “six constants of nature“? Why is it that no physicist thinks that?”
2. Where are the peer-reviewed scientific publications that show how fine-tuning has been “refuted by scientists again and again”?
3. Where are the peer-reviewed scientific publications that “only get [a] “narrow range” by varying one single constant“?
These seem to be simple factual claims of yours that Barnes is claiming you are wrong about. So far as I can tell, rather than 1. prove you’re correct by citing evidence in response, 2. retracting your original claim., or 3. showing how Barnes is misconstruing your claims, all you’ve done is simply ignore the questions. I find this, regretfully, rather similar in how you responded to Thom Stark during that back-and-forth. He too made several points that you were simply factually wrong–I can list them if necessary–and rather than respond or admit error, you simply opted not to respond. In your responses to Ehrman, you’ve been incisive and complete. I find those markedly different from your response on these topics–and I think Lowder would agree with me on this. To be perfectly honest–and again, I am fully willing to be shown wrong, as someone who finds your work often wonderfully edifying–I suspect you have wandered outside your area of expertise and are too stubborn to admit it.
Those questions from Barnes are over a year old, and, so far as I can tell, unanswered. I think we all would like to see your response, whether retraction or vindication of your claims, as well as the answers to the many criticisms he’s made in his most recent replies. My opinion, of course, doesn’t mean much, so I suppose we’ll have to wait for Lowder to opine once more. If he once again finds Barnes to be making devastating points, not indicative of a “kook,” maybe you’ll humor the suggestion that you may have made some mistakes here. Or, perhaps, the “kook” label will then be thrown at Jeff.
I never said there are only six constants of nature. I said Christian apologist James “Hannam also criticises [Stenger’s model] for only addressing four rather than six constants, but in fact only four constants are relevant for generating long-lived stars.” That’s Hannam, not me. I never said there were only six constants or even that there were six (and that was indeed fourteen years ago—I trusted Hannam then that there were about six; I know better now). Barnes links only to this article. So once again, he doesn’t even know what I’m saying, and gets it wrong, and over-relies on material over a decade old. You can see why I question his competence to have an intelligible debate.
I’ve answered the other questions. And Barnes has replied to that answer (finally; one of the few instances in which he has actually responded to something I actually said). I’ll discuss it soon.
Um, no. And you don’t have to check the math (well, go ahead if it pleases you.) The math is trivial. The answer is correct. I was just curious how you would calculate the uncertainties. Now I know.
Oh, good.
Yes, just run the equation separately for each margin. Then you get your range.
I discuss the method more in Proving History, index, “a fortiori.”
Hi Richard,
Richard: On those general questions, “[1] while Bayes Theorem is useful in “updating” probabilities, it is not necessary for assessing them at the outset” is not something a Bayesian would ever say,
Actually I have said this earlier and I am a Bayesian statistician :-). But it is also true! Bayes theorem (or the rules of probability theory) is a consistency requirement and not a way to assign probabilities from the onset. You even seem to agree on this point just a few sentences later :-).
As to “[2] that probability “measures information,” which can be but is not always in the form of frequencies,” that’s an assertion with no evidence to back it. I guarantee, anything he says about information from which he derives a probability, will always translate to a frequency of something. That consequence can’t be avoided. I think again he is confusing one kind of frequency statement for another, e.g. he is ignoring the fact that stating degrees of belief is making a statement about the frequency of being wrong.
Well, I would rather not get into this one again, but just for the sake of the casual reader I would like to point out that this particular interpretation of probabilities is your own and should be seen in contrast to all major Bayesian accounts of probabilities I am aware of. Perhaps you would agree that on the mainstream Bayesian view (for instance a Cox-axiom based view or a de Finetti gambling-based view), probabilities are thought to be a measure of states of belief/knowledge/information or some cognate thereof and only indirectly and not as a matter of their definition relate to frequencies?
I should add that Briggs seems to think this is my argument. It’s the argument of three expert mathematicians published separately in two sources. Which Briggs ignores and does not engage with at all.
I agree with the three articles from a formal point of view, but I have to disagree with your interpretation of these studies as I think they do not suppose the conclusion you draw. You can find my argument here:
https://www.scribd.com/doc/296697791/Richard-Carrier-s-rough-fine-tuning-argument
Cheers,
Tim
Hi Richard,
Bill asked how a person would typically perform inference when there was uncertainty about the probabilities, and I simply tried to give the answer suggested by Bayes theorem and what is done in practice in data analysis.
Re. “this is exactly what I recommend in PH (and run in parallel to the odds method in OHJ). The concern that our min/max is going to be too wide is a category fallacy. It is simply the case that in history, our margins of error often are that wide.”
As I said, I don’t want to comment on what historians ought to do or ought not to do, however in Bayesian data analysis you wish to see how your modelling assumptions (such as the uncertainty in the probabilities) translate into uncertainty in the quantities of interest. The way to do this is using Bayes theorem, and it will lead to different answers than the min/max approach — answers which from a Bayesian perspective is are spurious. You can say that since we are doing history we ought to do something else, however I was simply trying to provide the standard answer. If you are interested I can recommend guides for how parameter estimation is normally done using Bayes theorem?
Re. “I don’t fathom what’s ad hoc about calculating the a fortiori posterior and calculating the a judicatiori posterior. The full theoretical foundation for this comes from the philosophy of history and is detailed in PH. Quite simply, this is necessarily what you must do when the two questions you are asking are, “what is the highest probability of being true that I can reasonably believe this claim has?” and “what is the lowest probability of being true that I can reasonably believe this claim has?”
–
Well, by ad hoc I simply mean a method for manipulating probabilities that is not Bayesian. From a data-analysis perspective, you wish to know how probable different values of the quantities of interest are under the assumptions you have made (model assumptions, assumptions about uncertainty in probabilities, whatever) and Bayes theorem gives you a way to compute this. If you rather do the min/max procedure, you get numbers that do not represent this and are not computed using Bayes theorem. You can say this is what you want in history, but it is certainly not what you want in data analysis.
Cheers,
Tim.
Which is fine. I just don’t want anyone to mistake that for being the same thing as modeling and testing historical inferences.
You are describing a method correctly, that doesn’t pertain to ordinary inference procedures.
This is rather like teaching someone calculus, when they ask you how to do geometry. Or, like I said, advising someone commute to work on a three stage rocket, when walking will do.
Most inferences in philosophy and history (and indeed even sometimes in the sciences) do not benefit from extremely advanced and elaborate procedures. Yes, those procedures are useful, when you have the data that they are designed to get the best out of. But they are far too complicated for most purposes.
Reading Nate Silver on baseball in The Signal and the Noise, it was never clear to me that Silver’s methods could even ask the question, How meaningful is the notion of a “best” sports team?
I’m not clear on how Marilyn Vos Savant on the Monty Hall problem comes into it. But this is disconcerting because I could never understand any of her explanations. Nor could I ever understand how Bayesian reasoning does anything other than confuse the issues. For me, it was forgetting all the stuff about new information and updating that finally made it clear. Assuming random distribution of prizes, given enough trials, the frequency of any given door (left, right or center) hiding the grand prize would be 1/3. This real frequency does not change. The real frequency with which the grand prize would be found behind another door then is 2/3. That doesn’t change either. When Monty Hall opens one door, and offers a chance to switch to another door, the frequency with which the prize is found behind another door is still 2/3. Of course you switch. You can’t just update the information to think, two doors, therefore a frequency of 1/2 in a large series of random trials. Evidently what Bayesians means by updating is not so evident to some of us. Are we still Bayesians?
Prompted by this post I bought a copy of Jordan Ellenberg’s recent book and re-read the chapter on Bayesian inference. When Ellenberg declared that Bayesian reasoning leads you to take Nick Bostrom’s notions of simulated universes seriously, it inspires grave reservations. (This is similar to the recent declaration that Bayesian reasoning indicated some high probability the multiverse was real!) Bayes’ theorem is so easily read as frequentist, that it is tempting to read it as a necessary corrective to mechanical use of null hypothesis and p-values. But as someone observed (Mario Bunge I think,) observed, propositions don’t have probabilities. Assigning prior probabilities to the data set of evidence is the hard work of reasoning but it’s not clear how Bayes’ theorem helps this.
In a sense, you are right: Frequentism and Bayesianism are not so at odds as the battle-lines claim. But there are significant differences, which seriously matter (to life and limb even, as changes made in medical science and research due to introducing Bayesian reading have recently shown; but also to determining the actual rate of false positives in science research, which is being discussed everywhere in the literature now, as people realize the limited use of p-value reasoning). And there are general lessons one learns from thinking like a Bayesian, as it represents a complete logic, the entire machinery of a scientist’s inference from premise to conclusion, whereas frequentism by itself conceals much of that apparatus, leaving scientists prone to self-deception and error.
Steven, your approach is perfectly fine. In fact, it has been systematized in Gerd Gigerenzer’s book, “Calculated Risks”, where he calls this approach “Natural Frequencies.” I have used it (and his book) for many years in my honors class (for non-scientists) on Bayesian decision theory. I find that starting out from a “Natural Frequency” point of view is a very good way to introduce the ideas of Bayesian inference. In my class, which doesn’t use calculus, I can stick to finite state spaces so this works just fine. After getting the students comfortable with Gigerenzer’s approach and introducing decision trees, I can easily introduce the more conventional Bayesian notation and ideas.
The one thing that is different from frequentism is that Bayesians can regard probability even if the event is unique, i.e., not the result of something repeated over and over. For example, a given football game is played only once. It still makes sense (if you are into this sort of thing) to estimate probabilities of a given team winning this game and thus to make bets at certain odds (but not other odds) as rational bets.
P.S. For those interested, “Bayesians can regard probability even if the event is unique” also describes hypothetical frequentism. Which begins the debate over what the better way is to do hypothetical frequentism: by just adapting traditional frequentism to it, or going Bayesian on it. Of course, I think the latter.
I agree with Richard on this. I think that going fully Bayesian is the better way.
My use of Gigerenzer’s “Natural Frequencies” point of view is pedagogical. It is an easy way to introduce the ideas I want to present. One example is one that I often use when people ask me what I do and I want to give them an example of Bayesian stats (many will have had standard stats courses, which they hated)…I give them an example.
I tell them (correctly) that in 90% of the cases where a woman (in the general population) has breast cancer, a mammogram will come up positive, and in 90% of the cases where she does not have breast cancer, it will come up negative. I then ask, what if a woman has a mammogram and it comes up positive? How worried should she be?
As Gigerenzer points out, a large fraction of physicians will incorrectly say “90% probability she has cancer”, which is of course wrong. To answer correctly, you need the additional information that in this population about 1% of women have undetected breast cancer (the prior). Then I say, in a group of 1000 women, 10 will have cancer, and 9 of those cancers will be detected by the mammogram. Of the 990 women who do not have cancer, 99 false positives will occur. So in that group, there will be 99+9=108 positives, of which only 9 are actual cancers. So the proportion of women that test positive and have cancer is 9/108, or about 8%. The beauty of this approach is that I can do it in the parking lot, without even pencil and paper, and it will be understood.
Once the students understand this idea thoroughly, it is easy to transition to fully Bayesian calculations.
Agree with Richard’s comments. In particular, p-values are very problematic for a number of reasons. See:
http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
Also, I recommend following Andrew Gelman’s blog, it is one of the few things that I read faithfully. Andrew talks extensively about how things like p-values are misused. Type “garden of forking paths” into his search engine for example.
http://andrewgelman.com