I noticed some fake news spreading on Facebook last week. Like usual. But then I noticed what a great example it provided of how to defend yourself against just this kind of politicized fake news, with just a few basic principles of critical thinking. This time the source is some grifter and/or crankdoofus hunting for likes from stupid vegans who don’t know how science or math work.

I’ve done this before. There are lessons in From Lead Codices to Mummy Gospels: Essential Links on Dubious Tales and There Are No Muslim “No Go” Zones. And with respect indeed to math in Critical Thinking as a Function of Math Literacy and Innumeracy: A Fault to Fix. And in respect to the standard, sadly-too-often-needed refrain, “Please always check the sources before repeating that crap,” in On School Nutrition: How Republicans Create an Alternative Reality and Atheism Doesn’t Suck: How Science Does Not Prove Atheists Are Less Happy, Healthy, and Sane. And, of course, I’ve tackled vegan propaganda before, in Meat Not Bad and No, Bacon Is Not as Bad for You as Smoking. Which is not to say I think there is anything crank about being vegan or vegetarian (though they have their downsides, and they must be managed responsibly as diets if you plan to live that way); rather, it’s ideological propaganda pushing them I have an issue with. Like this article I’m talking about.

Today the target is a bullshit claim that vegans are smarter than meat eaters. It even throws in the claim that women are smarter than men, for good measure. And anyone who read that article should have known it was bullshit; yet they liked and shared it everywhere anyway. That people still lack the rudimentary skills to not be a vector for mind viruses is disturbing.

The Offending Article

For whatever reason, an article from two years ago on the website Dream Humanity (which doesn’t even have an SSL certificate) began being circulated repeatedly on Facebook last week: Vegans And Vegetarians Are More Intelligent Than Meat Eaters, Study Finds. The headline is false. But what’s more important to learn is why it’s false. Because that will teach you skills to arm yourself against any other deployment of the same scam tactics, by any other ideology or crankery, in aid of any other cause.

This article appeared then on what looks to me like a click-mill designed to rope in gullible liberals and ignorant hippies. You know, the kind of sappy eye-rolling garbage zine that covers the latest hot news in astrology. And still has six million followers. That kind of website.

Anyway. Note some key red flags you should have picked up on right away:

• It claims to be describing “a recent study” but never says where that study is, who its authors were, or when or where it was published.
• It claims this study found that “45%” of people are vegetarians or vegans. You read that right. Things That Make You Go Hmmmm.
• It claims this study proved “females are smarter than males.” (Oh, and also, it calls women “females.”)
• It never mentions the effect size, confidence level, or confidence interval for any of its numerical assertions.
• It’s full of highly confident opinions and value judgments for which no evidence is given (not even the “recent study”).
• It uses bold letters almost as often as a lunatic uses all caps.

It also seems English is not its author’s (?) first language. It starts right out with the phrase “a recent study made by scientists.” Made by scientists? It goes on to say “the vegans and vegetarians gained degrees and got high-powered job” (sic), “unlike others who’s IQ” (sic) “was lower and they haven’t changed their diet and lifestyle” (who “haven’t changed”?). Whoever this is, they may be a slick WordPress user, but good writer they are not; and journalist, they definitely are not.

Step One: Always Check the Sources

Never, ever trust an article about a study. Always find and read the actual study. And not just its abstract. You can be forgiven for not doing this when a study is reported on in a journal of very high quality (like the Washington Post or, indeed, Science News) and isn’t crucial to any decisions you are making or beliefs you are forming. But most of the time…yeah, check the study. This article at Dream Humanity gave little to go on. But there were enough keywords to ply on Google to find an article at PETA (yes…sigh…PETA) from, get this, 2006. Clearly about the same study; yet Dream Humanity was acting like it was reporting on something new—not a story that was already a decade old. So “a recent study” means, apparently, “more than ten years ago.”

At least PETA reported more responsibly, correctly describing the study, identifying their source, and even linking to it. Their account of it is simply to say that “individuals who were vegetarian by the age of 30 recorded an average of five IQ points higher than meat-eaters” (which is true—and not in the abstract, so they actually read some of the study) while rightly clarifying that the IQ tests in question were not taken when the subjects were 30, but when they were 10. In fact the study never says what their IQs were later in life, only at age 10. And PETA says that “after adjusting for social and economic factors,” the study found “the intelligent children were significantly more likely to become vegetarian later in life.” Which is not quite correct, but is what the study’s authors claimed, and the mistake being made here is commonly made even by top journalists; we’ll get to what the study actually said shortly.

That PETA article closes with a somewhat noncommittal but math-illiterate claim that that study’s results, by finding also more women are vegan or vegetarian, “may be” evidence as well that women are smarter than men. Which is telling, because this is something the Dream Humanity article also says—but is not in the study itself (whose authors would have known it wasn’t true)—so I think we’ve found Dream Humanity’s source. Yet they got wrong half of even what their source said.

Following PETA’s link gets us to the original BBC article on the study. And here, finally, we find some top notch science reporting. Right away it more accurately states the study found that “those who were vegetarian by 30 had recorded five IQ points more on average at the age of 10.” Like all real “news,” the byline tells us the year the study was actually done, the article names the article’s lead author, and the journal it was published in. So now we can find and read the original paper all this has been about: Catharine Gale et al., “IQ in Childhood and Vegetarianism in Adulthood: 1970 British Cohort Study,” in the British Medical Journal in 2007 (appearing a year earlier online). And this is what you have to always do these days, sadly. You can’t trust articles circulated on Facebook anymore. You have to fact-check them if their conclusions at all matter to you or any argument you wish to make.

The BBC article actually gives us numbers—which is important, because sample sizes matter. And this study had a really good sample size, in the high thousands. The BBC reports that “of 8,179” participants “366…said they were vegetarian” although “more than 100 reported eating either fish or chicken.” Ah. So, we aren’t actually talking about vegetarians, are we? A third of those reporting to be, weren’t. So the actual number of vegetarians we can now see amounted to barely 3% of the studied population (meaning, the UK). Although those “reporting” themselves as vegetarian amounted to 4.5%. Hmmm. Dream Humanity said 45%. Maybe they dropped a decimal? It seems more than a mere typo. Notably, no such statistic is reported in the PETA article. Or the BBC article. So the author at Dream Humanity got this “45%” (mistakenly from the actual 4.5%) from somewhere else. Mysteries abound. But in any event, we’ve confirmed the unreliability of our author.

Standard manuals state the standard error in IQ tests is itself 5 IQ points, and some studies show it could be as high as 15 points—I myself have varied by ten points even from just one day to the next, depending on which test I took. Meaning, any variance in IQ of 5 (or even 15) points or less is meaningless. You cannot say someone with 5 more points than you is “smarter” than you when the very next day the two of you can test again and score the other way around. Which happens so routinely, experts admit a difference of 5 points actually indicates no measurable difference in intelligence. Which is the final joke on that infamous racist manifesto The Bell Curve, which, after controlling for environmental factors like poverty, found no more than 5 points of difference between the races. So our hapless vegan here shares shelf space now with crypto-white-supremacists in making bullshit claims of differential intelligence. Well done.

Always Check the Effect Size and Confidence Interval

Whenever studies get reported, they often are presented as profound generalizations telling us a significant truth—when in fact the study found a weak to trivial effect, and could only claim that by expanding its error margins (their “confidence interval”) so wide as to tell us effectively nothing. The BMJ study is a classic example of this. We already observed its “effect size” was so small as to be below even the standard error of IQ tests. Which is already enough to dismiss its result as useless. Tons and tons of smart people eat meat; tons and tons of dumb people are vegetarians. Any correlation with childhood IQ is so weak as to be trivial.

The study’s authors do try to get around this problem by reframing their finding as a predictive “odds ratio,” claiming that high IQ in childhood nevertheless “predicts” vegetarianism in later life to some significant degree. But carefully note the actual result they claim here:

In multivariate analysis the odds ratio for being vegetarian was 1.20 (1.06 to 1.36) for one standard deviation increase in childhood IQ score. When the analysis was repeated after removing those who said that they were vegetarian but consumed fish or chicken, this result was essentially unchanged (1.19, 1.03 to 1.39).

So they are claiming that being smarter as a child makes it 20% more likely you’ll be a vegetarian later in life (the meaning of an “odds ratio” of 1.20). Note already this means most smart kids don’t become vegetarians; so intelligence is actually not leading people to be vegetarians. It’s just leading some of them to be, a mere 20% more than low IQ kids. This is already a pretty weak effect size, once again. “Most smart people reject vegetarianism” isn’t as catchy a title, of course. But alas, it is what their own data show.

Worse, though, is that to get this odds ratio, they have to soften their analysis so much that their error margin is enormous. In fact, they did not find an odds ratio of 1.20. First, they chose a 95% “confidence level,” because that’s popular, even though it’s widely recognized to be highly unreliable (physics requires a far higher confidence level, for example). Because even at best, 1 in 20 papers with this standard will give false results, and thousands of papers are published every year (you do the math). Reproducibility projects have found the frequency of studies at this standard actually holding up when the study is repeated is even worse, though, often almost as low as 1 in 3 (and this trend is not limited to psychology). But we’ll set all that aside (though we shouldn’t; science has a problem here, and it is high time it did something about it). What we should be most alarmed by is their confidence interval.

As you can see from the above quoted section, they only actually found there was a 95% chance the odds ratio was somewhere between 1.06 and 1.36, not that it “is” 1.20. The 1.20 is just the peak result inside that range. In actual fact, they cannot say the odds ratio was above even 1.06, a mere 6% increased chance of being vegetarian if you’re smart. So their data are consistent with the conclusion that almost all smart people reject vegetarianism! It gets worse. When we count only actual vegetarians (i.e. exclude those claiming to be who nevertheless eat chicken or fish), the confidence interval’s bottom drops to 1.03. So really, they cannot say the odds ratio was above even 1.03, a mere 3% increased chance of being vegetarian, a result so trivial as to be essentially, completely useless. When we actually look at what the study is claiming, childhood IQ has almost no proven correlation at all with adult vegetarianism.

Of course, even if it did, it would not follow that vegetarianism “is smart.” Higher IQ might correlate with it simply because higher IQ causes people to discover and contemplate more ways of choosing to live, and thus merely by chance you will expect more vegetarians among high IQ populations, along with more everything that deviates from the norm. Higher IQ also can increase the chance a person can figure out how to successfully maintain a vegetarian lifestyle, since it is intellectually challenging—it requires sharper grasp of nutrition, cookery, planning, experimenting, discovering flavorful and satisfying alternatives. Dumber people may simply be more set in their ways, less inclined to try difficult things, more prone to following what they were raised to prefer or conforming to their peers—more often, that is, than smart people. More often by a mere three percent? Even twenty? Easily.

Because in actual fact, high IQ is not the same thing as rationality. Smart people do tons of really dumb things (see The Difference Between Rationality and Intelligence and Why a High IQ Doesn’t Mean You’re Smart). In fact, high IQ can make you more prone to irrational decision making; and affords no protection against common causes of false belief formation (such as Myside Bias) or only weak protection at best. In general, as Mátyás Makay observes, “intelligence and rationality show only small-to-medium correlation.” So finding smart people doing a thing is not a reliable indicator that it should be done. So if that is what you wanted this study to find out, then IQ is simply measuring the wrong thing. This illustrates the importance of building your science literacy, to better evaluate link-bait headlines like “vegetarians are smarter than meat eaters.”

But alas all that is moot. Even after sampling thousands of people, these researchers still couldn’t confidently find anything more than a trivial 3% chance a smarter child will be a vegetarian later in life. Again, what they found, is that almost all smart children never become vegetarians. They are only trivially more likely to than dumb or average children. Which tells us nothing about the wisdom of being vegetarian—because that’s not what IQ measures.

Bad Math

The Dream Humanity article then goes on to make this brilliant argument:

Aside from showing that people who stopped eating meat and other animal products are more intelligent than meat eaters who refuse healthy diet and lifestyle, the study also showed that vegans and vegetarians were more likely to be female. This leads to the announcement of another important conclusion: females are smarter than males. Sorry guys. Go vegan.

Um. Okay. Math 101: it cannot follow from “vegans are smarter, and more vegans are women” that women are smarter than men. I mean, even apart from the fact that they aren’t: the actual variance is less than two or even one IQ points, and as we just saw, the error margin for IQ tests is between five and fifteen points, so a variance of a few points is meaningless. But even apart from that, the argument presented here is a non sequitur.

Think about an analogy: if red apples tended to be larger than green apples, and more red apples were harvested by women than men, it could still be that red apples harvested by women are slightly smaller than red apples harvested by men, but still larger on average than green apples. It could also be that the red apples women harvested were smaller even than green apples on average, but the apples harvested by the men were on average large enough to pull the average up for all red apples over green apples. And so on. There are several possibilities. Which is why you should never draw the kind of inference this doofus drew. Data can’t be used that way.

Do you know what you could do? Actually read the study. Which this doofus evidently didn’t. What does it have to say about its sample’s IQ breakdown by gender? “Men who were vegetarian had an IQ score of 106, compared with 101 for non-vegetarians; while female vegetarians averaged 104, compared with 99 for non-vegetarians.” So, actually, women are dumber than men, whether they eat meat or not. If you think two IQ points counts as dumber. Which would be dumb. Remember what we just said about the error margin for IQ tests? Oh, right.

This actually exposes a problem with the original study, and not just this doofus’s argument from it. Other studies from that same decade (which I just linked to above) found women test on average one or two IQ points above men. So how could this study’s sample have women testing one or two points below men? This suggests there could be something biasing the sample this study used, and indeed biasing it specifically in respect to the variable of IQ, which calls into question even its insignificant findings. Which brings us back again to the importance of checking claims against wider findings in the sciences on the same fact, and reading the original study yourself, instead of some hack’s crappy summary of it.

The BMJ study’s own authors responsibly warn that “the participants at the 30 year follow-up did gain significantly higher IQ scores at age 10 than those who did not take part,” even if only by a little, meaning there was a discrepancy between people who took IQ tests at age ten and didn’t participate in the study at age 30, and those who did: the latter were, for some reason, the higher scoring of the two populations. They speculate that “unless the relation between childhood mental ability and vegetarianism is in the opposite direction in non-participants, little bias will have been introduced in our study.” But if the variance they measured between men and women was the reverse of other studies of gender differences in IQ, it actually is possible the relationship between IQ and vegetarianism was also reversed in the non-participating group, by an equally trivial amount. Or, of course, the disparity could simply be explained by these tests being taken in 1970. Studies show that rising gender equality has closed any previously measured gap in IQ between women and men (the studies I cited earlier are all from thirty years later).

This is the problem with making a big deal out of what are in fact meaninglessly trivial differences in IQ in the first place. Hence my number one advice every time stats are thrown at you: don’t trust a headline; always find out what the effect size was (and attend to the confidence interval; it will hide the real effect size). If the effect size is trivial, toss that study in the trash and move on. It’s completely useless.

The Punchline

There is an extra fun joke in that last quote from Dream Humanity. Did you notice it said this result, of “vegans and vegetarians” being smarter, suggests you should “Go vegan”? The Dream Humanity piece repeatedly pairs and thus conflates “vegans” and “vegetarians.” In fact the study it claims to be summarizing only substantially speaks of vegetarians, and a third of them weren’t even real vegetarians. But it’s worse than that. What did the BMJ study itself say about vegans and intelligence? “On average, vegans had a childhood IQ score that was nearly 10 points lower than other vegetarians.” That’s right. Vegans were dumber. In fact, they were the dumbest population measured. If you think a variance of 10 points makes you dumb. Which, as I already noted, is a dumb argument. But it is Dream Humanity’s argument. So it’s very humorous to see how badly their own argument kicks them in the face here.

The BMJ study found vegans had a mean IQ of about 95, “compared with 104.8” in vegetarians. They note their thousands of participants only included nine vegans so this result is attended by much more uncertainty. But still. Remember what we saw above? Meat eaters, men and women, had average IQs of 101 and 99. Which means vegans were dumber than meat eaters—by about the same amount as the same study claims vegetarians were smarter. So this whole time, the Dream Humanity author is using this study to push veganism (it’s “Go vegan,” after all, not “Go vegetarian”), when the same study’s conclusion was exactly the opposite for veganism. Enjoy a good chuckle at that. And remember this as a key example of why you need to always read the actual study.

Conclusion

That Dream Humanity article everyone was meming and sharing and liking on Facebook got almost every single thing factually wrong. It claimed “vegans and vegetarians” were smarter; in fact vegans were dumber, only vegetarians smarter, and a third of whom weren’t even vegetarians; and they were smarter only at age 10, not when they were asked about their diet 20 years later; and they were smarter by a meaningless, trivial amount, below even the error margin of IQ tests themselves; and the same data show almost all smart people actually rejected vegetarianism. It also falsely inferred more women being vegetarian meant women were smarter than men; in fact, the study’s data found they weren’t, but were actually dumber than men, though by again a trivial amount, which has since reversed (as women now test just as trivially above men in mean IQ)—which is actually evidence against intelligence being a causal factor in vegetarianism. The article also confused IQ with wisdom or rationality, when in fact we know IQ measures neither.

Lessons to learn here:

• Always read the actual study
• Always check the actual effect size
• Always check the confidence interval
• Always check what is actually being measured, and when
• Always think through what all of this actually means
• Keep developing your science literacy to evaluate studies smartly

Please keep all that in mind, so you can help stop the spread of stupid crank memes like this. And help others learn why it’s important to stop liking and sharing garbage articles that misinform the public.