What is the Best Way to Combat Confirmation Bias?
If I was 20 years younger and participated in a certain body art trend, I might have a tattoo inscribed on my forearm that said something like this:
Confirmation bias MT @jayphilips: “The assumptions you start with dictate the conclusions you arrive at.” @james_christie #CAST2014
— Josh Meier (@moshjeier) August 12, 2014
As the Skeptics Dictionary notes:
Confirmation bias refers to a type of selective thinking whereby one tends to notice and to look for what confirms one’s beliefs, and to ignore, not look for, or undervalue the relevance of what contradicts one’s beliefs.
This is a very human tendency. Public debates on contentious topics, such as GMOs and climate change, are rife with confirmation bias. I know that everything I write on such topics is viewed by most readers with preconceptions, which include strong opinions already held about the topic itself, or even about me. I’m not really bothered by this, because I don’t see myself as a persuader. I’m not looking to change minds or win you over to any one side of an argument (even when I aim to debunk myths and misinformation). I’m much more interested in chronicling and exploring the contours of a particular scientific dialogue or narrative–how it formed, how it’s maintained, who is shaping it.
If, as a result of this, you come to reexamine some of your own assumptions, well, that’s an added bonus.
So one of the things that’s fascinating to me about confirmation bias is how it manifests itself in media and in those who probably think they are not infected by it.
For example, the other day I saw a story in a Scottish newspaper that reported:
Tens of thousands of Scots may be suffering from a hidden sickness epidemic caused by wind farms, campaigners have warned.
Now where have I seen a story like that before? Oh yeah! Anyway, I’ve previously looked into this new assortment of medical ailments characterized as Wind Turbine Syndrome. It is, as the Colbert Report noted, an oddly contagious syndrome. Perhaps unsurprisingly, the Daily Caller, a conservative media outlet, picked up on the Scottish article. I say unsurprisingly because if you plug in “wind turbines” into the Daily Caller’s search engine, you’ll see a gazillion articles about their various dangers and the over-hyped claims from greens about wind energy. So it makes sense that a new story about how wind turbines are sickening thousands of people would find its way onto the Daily Caller website.
This made me curious how the same media entity would view fracking, which is a form of natural gas extraction that many in the green community regard as harmful to human health and the environment. (Some might say the anti-wind and anti-fracking campaigns are tactically similar. Campaigners on both sides have been accused of embellishment and scaremongering.) If you plug in “fracking” in the Daily Caller’s website, you’ll see a gazillion articles downplaying or casting doubt on any studies that found fracking to be harmful.
In sum, the glories of fracking are celebrated at the Daily Caller, while the hazards of wind power are played up. What is going on here? Again, from the Skeptics Dictionary:
Motivated reasoning is confirmation bias taken to the next level. Motivated reasoning leads people to confirm what they already believe, while ignoring contrary data.
Examples of this abound on Twitter, such as the one case I discussed about the energy writer who has become fixated on wind turbine syndrome (and seemingly unmoved by any concerns about fracking).
To be sure, confirmation bias is not exclusive to any one political persuasion. Prominent skeptics with a liberal worldview can be just as credulous.
Of course, it’s always easier to see this in others and not ourselves. I freely admit that I have my own biases and assumptions that I struggle to keep in check. But as a journalist it’s incumbent on me to be mindful this. One way of combatting biases, I’ve found, is to stay in a constant state of intellectual flux, so I can be open to new information and perspectives. Another way is to have expert sounding boards that I trust. These are people whose scholarship I (and many of their peers) respect and whose even-keeled disposition I admire.
I’d be curious to hear how you combat your own biases and assumptions on topics that have become ensnared in highly charged debates.
UPDATE: This 2013 essay is a thoughtful take on wind turbine syndrome. The author, who has left a comment on my post, is employed by the renewable energy industry.
If I go to Drudge I immediately follow it with Slate.
If I go to Foxnews.com I immediately follow it with CNN.com.
During election season I watch a different network for coverage for each respective presidential debate.
etc…
As I consume each source I do so with conscious knowledge of their slant. By doing this I can see how the “journalists” and pundits subtly, and sometimes not so subtly, inject their biases into their reporting. It’s amazing to see the glaring difference in what parts of a story are highlighted and what is ignored or buried. Sometimes what’s not said is the most telling part. And, with that, it helps me spot biased reporting so I can filter out the crap that would otherwise confirm my biases and assign the proper weight to things that go against my biases.
I also like to use sounding boards on blogs like this. Yours has some of the best people to interact with. (although I do miss BBD for this purpose)
And, BTW, you are always a refreshing break from the norm of biased “journalism”. I can call you a journalist without feeling the need to put quotes around it…
Thanks for the kind words.
Yes, a diverse media diet is essential. But I quibble with the Drudge/Slate equivalency. Sure, Slate has a liberal slant, but it’s really an imperfect comparison. I think I would point instead to the Huffington Post, which, BTW, was conceived as the liberal counterpart to Drudge, who in its prime had a lot of influence.
Yes. Drudge/huffpo is a better comparison. Slate does leave some room on the page for other viewpoints.
I rarely go to huffpo anymore. My doctor said I need to lower my blood pressure, and as a conservative, I just find the site maddening (as I’m sure a liberal would find drudge). So I decided to stop going to huffpo and keep eating cheese steaks. I’m happy with that decision… 😉
Sometimes events occur that bring a bias out into the sunlight.
An example I remember is that a paper came out claiming a Hockey Stick using paleo data from Australia/Southern Hemisphere. McIntyre examined the paper and found they didn’t actually execute the math as shown in the paper, and when the the math was done as shown in the paper….no hockey stick. The paper was retracted.
They could proceed:
1. Use the math as shown in the paper, and present the results as per McIntyre. I assume not finding a HS should be considered “good science”.
2. Go back and change the math so a HS result would be produced.
A long story short, they chose #2 and resubmitted the paper. The journal rejected the re-submission (as they should have, good for them). The paper was dropped as far as I know.
Well they put themselves in quite a fix here, and their actions post-retraction demonstrated an unambiguous confirmation bias problem.
I realize this story touches on inflammatory subject matter, but the specifics are not the point, it is showing how confirmation bias is occasionally testable in the real world.
It is well-nigh impossible to keep the biases out of the human “reasoning” process but there are inspirations out there; for me it is Bob Edwards, ex of Morning Edition on NPR. By the way NPR, you have turned out to be a big, big disappointment for journalistic integrity………..
I recall exactly when I fell in love with Mr Edwards – it was when VP Dan Quayle, making a speech which highlighted that kids do better in married homes, said: “It doesn’t help matters when primetime TV has Murphy Brown, a character who supposedly epitomizes today’s intelligent, highly paid professional woman, mocking the importance of fathers by bearing a child alone and calling it just another lifestyle choice.”
Man did Quayle take a lot of heat for that! And there were very few defenders of Quayle to be found in the media……but there was NPR Bob Edwards, going out of his way to say that he agreed with Quayle on this point. This, despite the fact that anyone listening to NPR knew he absolutely was not a fan of Dan Quayle. As Edwards said later: “Quayle’s a dunce, but on this one point, he’s right. Fathers have always been dolts on TV. We’re irrelevant, superfluous. So the latest twist is just to feed the fiction that children don’t need fathers.”
You rule Bob Edwards……….
To control my biases I think things through and stay focused on basic principles. If it violates a basic principle, it’s probably wrong until proven otherwise many times.
I don’t put any faith in any media outlet or make any effort to balance left and right perspectives. To me that’s kind of like saying “well, I have bad data from these people and bad data from those people, maybe they’ll cancel each other out and give me the right answer”. Zero chance of that.
But the idea that a person can be an open book, constantly just accepting “new data” at face value and updating all of their “beliefs” accordingly, like a spreadsheet updating your mortgage balance – well, that’s just flat out bogus. Can’t happen and it shouldn’t happen. Much of the “new data” turns out to be wrong in the long run. Confirmation bias is as much our friend as it is our enemy.
People’s thinking and ideas evolve slowly for good reason. If our thinking constantly accepted every new bit of data as true and real, we’d never know anything. The world would be a raging madness of contradictions contradicting one another.
I’m a little concerned about the idea that a diverse media diet cures the bias problem.
A better way to cure the bias problem is to get educated and use your education. IMO, it’s shocking how little people use what they “learn” in school and rely instead on what their friends and the media say to form their opinions – even people with advanced degrees. People with advanced degrees just have more sophisticated methods of denying they have any biases.
Keith, I think you relying on “trusted experts” is more or less the same as getting your opinions from your friends. Why should you trust these people any more than other experts? You say it yourself: you admire them. I’m not sure that’s the scientific way to go about it. The fact that you admire them doesn’t make them more likely to be right (or less likely to be wrong, I suppose, but either way not the best approach to find truth).
It’s a vexing question.
I try to be as conscious as possible of the fact that my own employment necessitates a certain worldview – I work in the renewable energy industry in Australia, so, perhaps the electrified meat between my ears is convincing me that, say, ‘wind turbine syndrome’ is mythical and unsupported by science, when really, it’s a catastrophic plague causing suffering in hundreds of thousands of people.
Generally, the best way to examine this is simply to defer to people who are less likely to be impacted by the same biases that I am – say, the Australian Medical Association, or the National Health and Medical Research Council, or state health bodies, etc etc.
These bodies tend to reach the same conclusion – there’s no evidence that their assertions of causality are correct.
So, I try and combat my own biases by deferring to experts. It works, usually.
I wouldn’t call the Daily Caller a “media outlet.” They are a political advocacy group; they make no attempts toward journalistic objectivity.
I read both HuffPo and Drudge, and the first upsets me as much as the second. Some days more. But neither upsets me as they once might have; you have to realize what game they are playing, the way they write headlines to pull in eyeballs, or to reinforce reader biases. Talk radio is the same thing — they purposely say very provocative things (whether they believe them or not) to manipulate listeners and draw attention to themselves. That then gives fodder for HuffPo, Drudge, Fox, MSNBC etc, which the talk shows then criticize, etc. It’s a circular food chain — all profit, but truth, and done best (Glenn Beck) it’s very lucrative. So IMO schools and universities need to teach much more about media criticism. I think people everywhere have been struggling with perspective since radio was invented. Or at least TV. Humans didn’t evolve to deal with these one-to-many information distributions, or, now, many-to-many, and I think they are fundamentally confusing, and alien and threatening to the human psyche that took several million years to develop.
I try to keep a distance, basically. I ask myself what if I had never heard of this before, what if I was somebody else, what if I wouldn’t have to justify changing my mind to anybody. It helps btw if people are used to you changing your mind. I generally think though that scientists don’t take cognitive biases seriously enough in their own work. I’ve written about this on my blog many times. They hold on to their convictions long after the promise of some theory has faded, thus the saying that progress is made on funerals and not on conferences 😉
Oh, and then there’s the bias blind spot of course which kinda makes all my efforts redundant I guess, so I often just tend to admit I’m biased. At least then people know what they get 😉
“I think people everywhere have been struggling with perspective since radio was invented. Or at least TV.”
Doubt it. Humans have been trying to BS other humans since they could talk and probably longer. The traveling snake oil salesman preceded TV by at least a few centuries. We’re the lying species – some of us are good at sussing out lies (wise), others aren’t so good at it (gullible).
Or, giving the benefit of the doubt, and keeping with the theme of the program, you could call us The Biased Species. We have a strong tendency to believe in what benefits us. When we believe in things, we exaggerate their efficacy in discussing them with others. We’re not really lying, at least not in our own minds, because we believe what we’re saying.
I certainly do understand the game. I just find the game amusing when it’s supporting “my side” and annoying when it’s the other (tribalism no doubt…). But both sides certainly play the game and it’s, IMO, the primary reason for the partisan issues clogging up our government today.
Once you understand the game, you can glean much information from things that aren’t said. Such as, if you go to Foxnews site and CNN site on a slow news day you’ll see completely different stories “above the fold” and prominently displayed with images. The non-news stories they promote help you to understand what games each side is playing.
You can also get information based on how long stories live. Foxnews humped Benghazi every day for about 2 months, you had to search CNN to find stories about it. These types of things are very telling, and the behavior of the news outlets tells you as much about the politics involved as any words in the story could.
I’m certainly not condoning it, I hate that it happens to the point that I think it may need regulation. And that’s coming from one with a libertarian bent!!!
“I’m a little concerned about the idea that a diverse media diet cures the bias problem.”
It’s likely not a cure, but it’s certainly better than just going to places that confirm your biases.
Some of us worried more about alcohol and pot than learning anything in school (oh, if one could go back and do it over…) so we have to work with what we have. MSM does a good job of dumbing it down for the masses. But you need to get the dumb version from both sides to see the entire picture…
I’m not talking about BSing other humans, but the weirdlly different experience/perspective of having the world come to you (through a speaker or screen), of feeling like your some other place than you really are, of feeling like you’re part of the place on the screen, when you’re not part of it at all but just passively viewing it. Of thinking we knew Robin Williams because we had watched him for a few hours on a screen. Of thinking crime is increasing because the local news reports about it nonstop, when it’s actually decreasing. Of thinking the Iraq War wasn’t horrific because the media reported it that way and purposely did not show the gruesome suffering, death, and destruction. The medium is the message.
The best way to fight confirmation bias is with cognitive dissonance.
People either see their bias for what it is, or their head explodes.
“Keith, I think you relying on “trusted experts” is more or less the same as getting your opinions from your friends.”
What I actually said: “One way of combatting biases, I’ve found, is to stay in a constant state of intellectual flux, so I can be open to new information and perspectives. Another way is to have expert sounding boards that I trust.”
Your comment is a good example of how bias colors your interpretation of what I said. For I don’t see how you get “relying” on trusted experts when what I actually said was using them as “sounding boards.”
Additionally, I’m a journalist, not scholar or scientist, so I have to talk to experts in the fields I write about. This doesn’t mean I don’t read the literature or science–it means that I talk to experts, which is what all journalists do. The point I was making is how I come to trust some experts more than others. And I gave that criteria.
And when you say I “admire them,” that’s also misleading, because what I actually said is that I admire their even temperament. That tells me something important.
I also look for experts that don’t twist the meaning of what someone says.
That might work with some people but not with ideologues. They just completely ignore the rational, factual content of what you’re trying to point out and attack your personally: you’re either crazy, stupid or wicked, or a combination thereof.
Pay attention to any criticism of one political ideologue by another and that’s always the underlying theme – GMO’s, climate change, immigration, tax policy, you name it.
I encourage those types to elaborate on their beliefs, exposing their mental illness for the world to see.
From inside the GMO world—the best way is to surround yourself with people who challenge you—and most importantly, challenge your data. The fact that scientists who work together spend far more time questioning each others experiments than they do plotting world domination is completely lost on the anti-GM types.
I read a lot of stuff on the web, more conservative than liberal, and a lot about climate change. I subscribe to two newspapers. This post is the first time I’ve seen mention of Wind Turbine Syndrome.
a.k.a. yuppie hypochondria. It’s like when we were kids, millions of children went through grammar school eating peanut butter and jelly sandwiches every day; I don’t remember anybody getting sick or dying. Now they can’t even bring a peanut butter sandwich to school without coming home with an indignant letter from the principal.
Speaking of confirmation bias:
https://twitter.com/nntaleb/status/498106279861444608
https://twitter.com/nntaleb/status/497053419275034624
https://twitter.com/nntaleb/status/494643664682221568
https://twitter.com/nntaleb/status/493740175269572608
Blaring headlines that misrepresent issues in order to gain an audience is a form of BSing. I don’t think it’s particularly new (see PT Barnum). What’s new is how easy it is to cross-check the factualness of the information.
“Some of us worried more about alcohol and pot than learning anything in school…”
🙂 Too funny.
Interesting thoughts. I’m a little surprised that you presume I’m intentionally twisting what you said. I don’t misrepresent what people say intentionally. I think if that’s your perception, perhaps your biases are playing a role in that perception.
On reading your comment, I think I do see some of my own biases reflected in how I read what you wrote, but I also think I see some hair splitting in your criticism.
AS for my own biases, we all know that the general population relies much more heavily on opinions of people they know than on factual information. But I think it’s much more common in the scientific community and among supposed “thought leaders” – people that are expected to provide independent opinions – than claimed. So, yeah, I guess I jumped on that.
But I also think there’s substance to my point. If you’re selecting your opinions from people with even temperament, people you trust, and people with characteristics you admire, aren’t you using the same criteria that people use to select friends? So I think you’re hair splitting by claiming there’s some subtle shade of meaning that I overlooked. Perhaps there is, but is it relevant?
I value your work. To me there’s not much worth reading in the MSM and your blog stands out as one of the more thought provoking blogs on the web. But I still think you come with plenty of biases – just as we all do.
“I’m a little surprised that you presume I’m intentionally twisting what you said.”
I didn’t say “intentionally,” and I don’t think that at all.
In any case, rather than engage in more “hair splitting,” as you put it, I’ll just end this exchange by agreeing with you that I have biases. I think I owned up to that in the post. I’m sure they are still going to still influence what I write. I just wanted to share how I go about trying to lessen that.
Lastly, thank you for the kind words. I really do appreciate hearing how someone who probably disagrees with me on many things still values what I write.
I agree with you on some things and disagree with you on others. I don’t have much to say when I agree. I’m not really the cheerleading type.
I challenged you – rightly or wrongly or obtusely – on your biases, but I still think you’re one of the least biased writers around. One way to keep yourself honest in the face of the tendency to be less objective with the people you admire/trust/etc is to challenge those people (as Loren Eaton suggested).
I’ve suggested many things for you to write about. I do that because I value your views and I’m interested in your take on those topics. No, I won’t agree with you 100% on anything, maybe not even 50%, but I almost always find some nugget of perspective in your work that’s novel and important enough to reshape my views, even if it’s just a shade.