How to detect baloney the Carl Sagan way | Michael Shermer — In 1995, just a few months before his death, astrophysicist Carl Sagan published The Demon-Haunted World: Science as a Candle in the Dark. In that book, he wrote a chapter called 'The Fine Art of Baloney Detection', and from it sprang what skeptics call the 'baloney detection kit'. This is a set of tools for critical thinking that has continued to develop since Sagan's death, 22 years ago. Here, skeptic and science writer Michael Shermer explains key lessons from Sagan, and from his own college freshman course 'Skepticism 101', where teaches students ten basic questions that will help them debunk untruths, and call out baloney when they see it.
ack in the late '90s we introduced the Baloney Detection Kit, inspired by Carl Sagan’s 'Demon-Haunted World' where he had a chapter on the Baloney Detection Kit. He had his set of questions; I kind of developed my own because I started encountering other people that disagreed with me, you know, “we never went to the moon” people, conspiracy people, whatever, and I thought okay so: How do we know—if I don’t know what’s coming down the pike in ten years from now, if I am going to teach my students how to think critically, what are the key points, like just basic questions they can ask?
So it begins with one: how reliable is the source of the claim? Here’s the claim, how reliable is it? What’s the evidence for it? What’s the quality of the evidence? Where does it come from? Who said that? Is this some fake news alternative site thing or is it the Wall Street Journal or The New York Times? I mean, the source really matters.
Has anyone tried to disprove the claim? This is super important because everybody thinks they’re right and every website has testimonials about this product or that idea; the question is not what do your supporters think but what do the people who don’t agree with you think? Because that’s what I want to know.
Has anyone run experiments to try to disprove your theory? In science, this is as basic as it gets. Karl Popper called this the principle of falsification, that is we can’t ever prove a theory correct, but we can disprove it by having an experiment that shows it’s wrong.
If you can’t falsify it, what are you really doing? And my favorite story on this, by the way—let me just have a little sidetrack here from Carl Sagan, he’s got this great little section in his book 'Demon-Haunted World': “There is a dragon in my garage. I have a dragon in my garage. Do you want to see it? Let me show you.” So I pull up the garage door I go, “Look. Can you see the dragon?” And you look in there and you go, “I don’t see anything.”
“Oh, sorry, this is an invisible dragon.”
“An invisible dragon?”
“Yeah, yeah he’s invisible.”
“Well, what if we put some flour on the ground and then we’ll get the footprints of the dragon.”
“Well, no, see, this is a special dragon that hovers above the ground, it floats. It’s an invisible floating dragon.”
“An invisible floating dragon. Okay. Wait, I have some infrared cameras here we can detect the heat of the dragon.”
”No, see this is a cold-blooded dragon. It doesn’t give off any heat.”
“What about the fire? We can detect the fire that the dragon spits out.”
“No, it spits out cold fire.”
You see the problem? If there’s no way for me to falsify that there’s a dragon there, what’s the difference between an invisible floating heatless dragon and no dragon at all? None.
And of course we can apply this to god or any other supernatural/paranormal-type phenomenon. If I can’t debunk it, if I can’t falsify it, if there’s no way to test it, then how will we ever know it’s true?
This is the core of the Baloney Detection Kit. We have to be able to get to whether it’s true or not in some way, so it’s not just my opinion versus your opinion and we shout at each other.
Then we want to know: does the claimant’s personal belief somehow enter in? Because of course we all have personal opinions and beliefs about things; my politics, my religion, my ideology can influence me. It doesn’t make it wrong, but it’s good to know if somebody has an agenda. So when you watch Fox News you know that they have an agenda for sure, or there are other sources on the left that have a liberal agenda, maybe NPR, who knows. But you see it’s good to know that just in case, so when you hear the fact you go, “Well maybe, but I know this guy has an agenda.” So that’s the kind of thing.