Where Science is Flawed
In its current issue, The New Yorker has an excellent piece on the prevalence of (unconscious) bias in scientific studies that builds on this recent must-read piece in The Atlantic. And to some extent, Jonah Lehrer’s New Yorker article builds on this story he did for Wired in 2009. Anyone interested in the scientific process should read all three, for they are provocative cautionary tales.
Back to Lehrer’s story in The New Yorker. I’m going to quote from it extensively because it’s behind a paywall, but I urge people to buy a copy of the issue off the newsstand, if possible. It’s that good.
His piece is an arrow into the heart of the scientific method:
The test of replicability, as it’s known, is the foundation of modern research. Replicability is how the community enforces itself. It’s a safeguard for the creep of subjectivity. Most of the time, scientists know what results they want, and that can influence the results they get. The premise of replicability is that the scientific community can correct for these flaws.
But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology.
This is not the same as scientific fraud, Lehrer writes:
Rather the problem seems to be one of subtle omissions and unconscious misperceptions, as researchers struggle to make sense of their results.
He then describes “one of the classic examples” of selective reporting:
While acupuncture is widely accepted as a medical treatment in various Asian countries, its use is much more contested in the West. These cultural differences have profoundly influenced the results of clinical trials. Between 1966 and 1995, there were forty-seven studies of acupuncture in China, Taiwan, and Japan, and every single trial concluded that acupuncture was an effective treatment. During the same period, there were ninety-four clinical trials of acupuncture in the United States, Sweden, and the U.K., and only fifty-six percent of these studies found any therapeutic benefits. As [University of Alberta biologist Richard] Palmer notes, this wide discrepancy suggests that scientists find ways to confirm their preferred hypothesis, disregarding what they don’t want to see. Our beliefs are a form of blindness.
Lehrer then introduces Stanford epidemiologist John Ioannidis, the star of the The Atlantic story. Lehrer writes:
According to Ioannidis, the main problem is that too many researchers engage in what he calls “significant chasing,” or finding ways to interpret the data so that it passes the statistical test of significance–the ninety five percent boundary invented by Ronald Fisher. “The scientists are so eager to to pass this magical test that they start playing around with the numbers, trying to find anything that seems worthy,” Ioannidis says. In recent years, Ioannidis has become increasingly blunt about the pervasiveness of the problem. One of his most cited papers has a deliberately provocative title: “Why Most Published Research Findings are False.”
The problem of selective reporting is rooted in a fundamental cognitive flaw, which is that we like proving ourselves right and hate being wrong. “It feels good to validate a hypothesis,” Ioannidis said. “It feels even better when you’ve got a financial interest in the idea or your career depends on it. And that’s why, even after a claim has been systematically disproven”–he cites, for instance, the early work on hormone replacement therapy, or claims involving various vitamins–“you still see some stubborn researchers citing the first few studies that show a strong effect. They really want to believe that it’s true.”
That’s why [UC Santa Barbara cognitive psychologist Jonathan] Schooler argues that scientists need to become more rigorous about data collection before they publish. “We’re wasting our time chasing after bad studies and underpowered experiments,” he says. The current “obsession” with replicability distracts from the real problem, which is faulty design…In a forthcoming paper, Schooler recommends the establishment of an open-source database, in which researchers are required to outline their planned investigations and document all their results.”I think this would provide a huge increase in access to scientific work and give us a much better way to judge the quality of an experiment,” Schooler says.
As I said, you really should read the whole piece if you want to learn more about this widespread but little discussed problem with a key tenet of the scientific method. Lehrer perceptively concludes:
We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.
UPDATE: Jonah Lehrer’s New Yorker article has spawned much discussion in the science blogosphere. See Jerry Coyne, Randy Olson, Steven Novella, John Horgan, Matthew Nisbet, Charlie Petit, David Gorski, and Judith Curry. Additionally, Lehrer, at his blog, elaborates on what his article is NOT implying.
UPDATE: Five days after hitting the send button on my post, I see that Marc Morano has linked to it. Readers coming here via Climate Depot should be aware of Jonah Lehrer’s answer to a reader who asks: “Does this mean I don’t have to believe in climate change?” Lehrer’s response:
One of the sad ironies of scientific denialism is that we tend to be skeptical of precisely the wrong kind of scientific claims. In poll after poll, Americans have dismissed two of the most robust and widely tested theories of modern science: evolution by natural selection and climate change. These are theories that have been verified in thousands of different ways by thousands of different scientists working in many different fields.
I concur with Lehrer’s assessment of the science underlying evolution and climate change.
This is an excellent write-up. Thank you for the links and excerpts.
But what relevance does this have to the settled science of climate change and the fact that Naomi Oreskes a science historian, never found a single peer reviewed paper that disagreed with the consensus?
Are you really suggesting that “chasing significance”, publish or perish, and a scientist’s career and life choices might somehow consciously or not, change the results of an experiment or what gets published? As if scientists also wanted nice jobs, nice houses, department chairs, and control over their labs.
Nice try, Keith.
What’s all this fuss I hear about replication? Just re-run the model!
“It didn’t turn out that way. In poring over medical journals, he was struck by how many findings of all types were refuted by later findings. Of course, medical-science “never minds” are hardly secret. And they sometimes make headlines, as when in recent years large studies or growing consensuses of researchers concluded that mammograms, colonoscopies, and PSA tests are far less useful cancer-detection tools than we had been told; or when widely prescribed antidepressants such as Prozac, Zoloft, and Paxil were revealed to be no more effective than a placebo for most cases of depression; or when we learned that staying out of the sun entirely can actually increase cancer risks; or when we were told that the advice to drink lots of water during intense exercise was potentially fatal; or when, last April, we were informed that taking fish oil, exercising, and doing puzzles doesn’t really help fend off Alzheimer’s disease, as long claimed. Peer-reviewed studies have come to opposite conclusions on whether using cell phones can cause brain cancer, whether sleeping more than eight hours a night is healthful or dangerous, whether taking aspirin every day is more likely to save your life or cut it short, and whether routine angioplasty works better than pills to unclog heart arteries.”
Heh. Anti-vaxxers the lot of them.
I’m in awe, Keith. I’m utterly lost for words. It’s like a dream where I’m in an echo chamber, and I’m a duck.
So now you’re the William Tell of science journalism, Keith? In your case, I’m afraid the kid didn’t make it.
I kept looking for you to note that the focus of the hero (jeez, is this the kind of rhetoric you teach your students?) is on the medical end of things. But this was a drive-by, so collateral damage was part of the point.
BTW, in the aggregate, the acupuncture results do tell us something: That any effect from it is small and hard to detect.
I’m sorry, you said star, not hero. The point stands.
Steve Bloom,
I’m not sure what your point is regarding the focus of the hero. But the Atlantic article does go on to claim:
“Medical research is not especially plagued with wrongness. Other meta-research experts have confirmed that similar issues distort research in all fields of science, from physics to economics (where the highly regarded economists J. Bradford DeLong and Kevin Lang once showed how a remarkably consistent paucity of strong evidence in published economics studies made it unlikely that any of them were right). ”
Steve, would you clarify your point?
Steve, what are you talking about? What is your complaint (this time)? Be specific. Is is that you don’t buy the premise of the New Yorker and Atlantic piece? Or that I should have mentioned that the medical field is the focus of the The Atlantic piece? Didn’t I provide a link? Was I talking about that story?
I’ll tell you what, though: I’ll play along. Do you think the problem identified in the New Yorker piece is limited only to the medical, ecological and psychological sciences (the latter two are fields the Lehrer story focused on)?
The original research paper , Why Most Published Research Findings are False, is also well worth reading. Ioannidis picks examples from his own field of medicine, but the structure of his argument is certainly applicable to e.g. climate science.
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124
Some highlights:
“Corollary 2: The smaller the effect sizes in a scientific field, the less likely the research findings are to be true. Power is also related to the effect size. Thus research findings are more likely true in scientific fields with large effects[…]In the same line of thinking, if the true effect sizes are very small in a scientific field, this field is likely to be plagued by almost ubiquitous false positive claims.”
“Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true. Flexibility increases the potential for transforming what would be “negative” results into “positive” results.”
“Corollary 5: The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true. Conflicts of interest and prejudice may increase bias. […] Prejudice may not necessarily have financial roots. Scientists in a given field may be prejudiced purely because of their belief in a scientific theory or commitment to their own findings. […] Prestigious investigators may suppress via the peer review process the appearance and dissemination of findings that refute their findings, thus condemning their field to perpetuate false dogma. Empirical evidence on expert opinion shows that it is extremely unreliable.”
“Claimed Research Findings May Often Be Simply Accurate Measures of the Prevailing Bias […] the claimed effect sizes are simply measuring nothing else but the net bias that has been involved in the generation of this scientific literature. Claimed effect sizes are in fact the most accurate estimates of the net bias. It even follows that between “null fields,” the fields that claim stronger effects (often with accompanying claims of medical or public health importance) are simply those that have sustained the worst biases.”
HD, where’s the support for the claim about physics (which climatology is basically a sub-field of)? Note that we need more evidence than just the scattering of cranks that seems to be inseparable from the human condition. As well, it seems a little trivial to point out that in the physical sciences many results that add significantly to our understanding are somewhat incorrect in the sense that they’re partial. That by itself is no evidence of corruption or sloppiness. Nor is the fact that in fields where interpretation is difficult there will tend to be more corrections of past results. There’s a real risk here of confusing corruption and malfeasance with the ordinary scientific process.
The economics example is comparing fish to fowl. Economics is dismal, but it’s no science.
Keith, hopefully the above answered your questions, other than a) my snark about your over-florid rhetoric, which I don’t think requires discussion and b) the point that this being a blog whose primary topic is climate science and politics, and since not everybody will follow the links, indeed you should have noted that they’re not especially on-point in that regard.
Re #8: Yes, undoubtedly experimental physics is the worst of the lot. Defund them all, I say! Especially the German ones. 🙂
Keith, also bear in mind poll results showing where journalists stand relative to scientists in terms of public trust. Just sayin’.
Steve,
This blog covers many topics, from science journalism and climate science to archaeology and energy issues, among others.
Let me get this straight: So even though I didn’t mention climate science once in this post, I still should have told readers that climate science is exempt from the phenomenon discussed in The Atlantic and New Yorker stories? Because that’s what you mean by not being “on-point,” right?
You never cease to amaze me, Steve.
Physics, Medicine, … human endeavors, subject to the scientific method, both subject to falsifiability and replication of experiment.
Why do you think Physics would be different than Medicine? What are the structural differences? (And are you saying that physics experiments are easier to interpret than medical experiment?) Physics same as medicine with regards to nonsense papers or different: what is the null hypothesis? What is your presumption?
I also quibble a bit with your claim that climatology is basically a sub-field of physics. I am afraid it’s become more Schrodinger’s Cat than that. When physicists are skeptical of climate science, we are told that well, they don’t understand climate science, it’s an entirely different field of science, altogether. (It’s an entirely different field of science.) And now I’m being told it’s a sub-field of physics. How convenient!? It’s a root(2)/2 wavicle!
On the other hand, I am also told that much of climate science is apparently done without regard to empirical evidence, but done within the virtual space of computer models. Which then seems, on its face, harder to falsify, harder to ground in empirical results, harder to replicate, easier to bend, medicine or climate science?
I would actually be curious in a real analysis examining the question: which is more based on models, climate science or economics? Which is more falsifiable? Which has better controls? Which makes for scientifically better (clearer, easier to test) predictions? Is either climate science or economics a “science?”
Nobody has mentioned climate science in this context…. I wonder why?
Climategate e-mails did reveal that there is certain bias among the scientists.
I believe that most scientists strive for a balanced view. What I do find disturbing is the way evidence supporting a certain view is picked from studies while the the contrary evidence is ignored.
IPCC used Fischer et al. (2005) for their AR4 section on African agriculture:
Other recent assessments using the FAO/IIASA Agro-Ecological Zones model (AEZ) in conjunction with IIASA’s world food system or Basic Linked System (BSL), as well as climate variables from five different GCMs under four SRES emissions scenarios, show further agricultural impacts such as changes in agricultural potential by the 2080s (Fischer et al., 2005). By the 2080s, a significant decrease in suitable rain-fed land extent and production potential for cereals is estimated under climate change. Furthermore, for the same projections, for the same time horizon the area of arid and semi-arid land in Africa could increase by 5-8% (60-90 million hectares). The study shows that wheat production is likely to disappear from Africa by the 2080s.
What IPCC did not mention was this conclusion of the same study:
Effects of socioeconomic scenarios are substantial, and results vary in a range with lower and upper values corresponding toSRES B1 and A2, respectively. By 2080, BLS projectsglobal cereal-production in the range 3.7″“4.8 G ton, depending on SRES scenario. Production in the developed countries ranges 1.4″“1.6 G ton; thus BLS computes for the developing countries up to threefold increases in production from the 1990 baseline levels, with fivefold and higher increases projected for Africa in all the scenarios, as a consequence of the substantial economic development assumed in SRES.
Selective reporting is a form of cheating. No matter who says what Feynman was right and he knew better than just about anybody.
http://en.wikipedia.org/wiki/Cargo_cult_science
@ HD (12)
Economics are worse for multiple reasons – that’s the short answer. I don’t have time for a long one but consider this, for example. Climate models are based on mathematical description of physicall processes that we understand quite well on a micro-level. Navier-Stokes equations are derived from first principles. We just don’t know how to solve them, especially when coupled to everything else that happens in the system. Macro-economics supposedly describe collective behavior of millions of people. Unfortunately we don’t have a mathematical model even for a single person, and it gets only worse from there.
I don’t have access to the full article, but my initial reaction to this was, “No duh?” Honestly, the basic idea here is obvious. It’s the sort of thing you see in your day-to-day life.
One thing which always struck me as odd about science is there is no “right” and “wrong.” There is just “accepted” and “rejected.” Science advances by convincing people to believe things, not by being right. Being right may help convince people, but that is the only reason it matters.
Excellent article Keith, I’m growing fonder of your blog by the day. I have yet to read the articles that you refer to but look forward to doing so.
Here’s a link to an interesting study on the peer review system that was done in 1977:
http://pages.stern.nyu.edu/~wstarbuc/Writing/Prejud.htm
Certainly worth reading in regards to the topic at hand. I posted it at Judith Curry’s blog a while back and she seemed to agree.
Key quote:
“With our vast literature on information processing and social psychology, have we assumed that scientists are somehow unaffected by the processes which appear to be so common in other members of the species?”
“Climate models are based on mathematical description of physicall processes that we understand quite well on a micro-level. Navier-Stokes equations are derived from first principles. We just don’t know how to solve them, especially when coupled to everything else that happens in the system. Macro-economics supposedly describe collective behavior of millions of people. Unfortunately we don’t have a mathematical model even for a single person, and it gets only worse from there.”
Is this accurate Sashka? It seems like hand waving.
“Unfortunately we don’t have a mathematical model even for a single person, and it gets only worse from there.”
Do we have a 3 dimensional quantum mechanical description of the Helium Molecule? The C02 molecule? (I honestly don’t know.) A string theory model of the Hydrogen Atom? Water Molecule? C02 molecule?
And yet, we have Boyle’s law.
Maybe you’re right, but I think what you’ve displayed is your own ignorance of macroeconomics and bias against it.
Of course scientists have been aware for decades of dangers inherent in different types of research and experimental design. Some are inherently more messy than others. And yes, there is a ‘wow’ publicity factor to be wary of, and yes, some fields of science (I’m looking at you, clinical researchers) are notorious for their (mis)use of statistics.
I’d already read the New Yorker article, and the Atlantic article (I guess KK and I both subscribe to the same mags — do you get Harper’s too, KK?). What struck me in both was that the sciences described were mainly ones where it’s difficult to do rigorously controlled experiments and/or hope to get robust results. A *lot* of Lehrer’s examples came from psychology. Ionnadis focuses on medical research. Economics and ecology are also cited. (Lehrer, perhaps seeing this pattern, included one example from a ‘hard’ science, a measurement in physics that keeps getting weaker and weaker overtime, but I *think* that measuring something very small to begin with….? And how rife is this phenomenon is physics, generally, compared to the others?
Writing “other meta-research experts have confirmed that similar issues distort research in all fields of science, from physics to economics” strikes me as a bit of hand-waving. he didn’t really show much beyond the ‘soft’ psychological sciences, ecology (multivariate) , and medical research (which, involving human subjects, involves both multivariables AND ethical issues). I’d be interested to know what factors, if any, make a research field more or less prone to the phenomena that Lehrer and Ionnadis describe.
@ Brandon
Of course, “no duh” is the appropriate reaction.
A friend of my is a theoretical physicist in solid body physics. He told me about 15 years ago that about 90% of published results are wrong. I thought he was a bit cynical at the time. But maybe not.
Oh, and obviously, scientists does ‘get it right’, even if not right away, typically through repeated independent observations that either support the original finding or not — call it ‘regression to the real’. Happens in climate science too. But I see that baby-and-bathwater implication (humans are flawed, therefore science can be flawed, therefore climate science is all wrong) is already well represented on the comments here, from the expected sources.
Even mathematicians have intuition about some things that are they are quite certain are correct but are probably not provable.
From Professor Lipton’s blog:
Bombieri listened very politely, asked a question or two for clarification, and said, “yes that lemma is surely correct.” We were thrilled, since this would make our proof complete. I then asked him for a reference. He looked at me and said:
He had great intuition about primes, but proving certain results was then and still is today completely beyond anything anyone can do.
Sashka–
Navier-Stokes equations are derived from first principles.
Well…. not quite. The choice of stress tensor that is linearly proportional to the strain rate isn’t derived from first principles. Even to the extent that one can say it’s “derived” it’s an approximation. (So are Fourier’s and Fick’s laws.) So, the stress tensor bit is mostly an empirical observation for many fluids. It doesn’t work for things like paint.
But that’s a rather unimportant quibble. The more important issue is that climate models don’t solve the Navier Stokes equations. There simply isn’t enough computational power to permit them to be solved.
We just don’t know how to solve them,
We have no closed form solution for the Navier-Stokes. But computer models that treat the equations exactly exist. Look up “direct numerical simulation”. Climate Models don’t solve them because the flow field is too large.
Unfortunately we don’t have a mathematical model even for a single person, and it gets only worse from there.
FWIW: We don’t derive the navier stokes from a mathematical model of a single atom. That doesn’t diminis our confidence in the Navier-Stokes equations in situations were we can be confinden they apply.
KK: You may want to reconsider your title.
“Science” is a strong over-generalizatio, because different disciplines are different.*
That scientists are human is not news.
But social and biological sciences (especially medical research relating to humans) are inherently fraught with more difficulties:
-inability to replicate an experiment exactly, or with slightly different parameters;
-tiny sample sizes and heavy reliance on statistical analysis (lacking the interlocking structure of laws of physics, where conservation laws alone quickly rule out many possibilities.)
-difficulty with confounding factors, especially for small samples of highly-variable humans.
-inability to experiment on humans.)
Every human is different. Every CO2 molecule is the same (modulo isotopes and current energy state).
It is totally unsurprising that medical research generates wrong results … but even so, are the medical researchers wrong on smoking? If so, make sure kids take up smoking at 12. They have lower likelihood of disease than of having Conservation of Energy and quantum mechanics repealed, which is what it takes to make the basics of AGW go away.
Computers models are really good for reducing uncertainty and better understanding regional effects, and since there are $Ts at stake, the relatively tiny budget for models is a great investment. (Satellites cost much more).
While I used to be a supercomputing guy, and many climate model runs have been done on systems I helped design, Arrhenius got a pretty good approximation of the basics 100+ years ago with paper and pencil.
http://www.aip.org/history/climate/co2.htm
“But I see that baby-and-bathwater implication (humans are flawed, therefore science can be flawed, therefore climate science is all wrong) is already well represented on the comments here, from the expected sources.”
Curious. I don’t see that anyone has suggested anything close to that except for you.
What I perceive is a great deal of exceptionalism that appears unwarranted. “Sure, sure, doctors fudge their results, but not climate scientists!” “Sure, sure, doctors want promotions and publication and better careers, but that’s because medicine is soft science, climate scientists operate in the real world and our virtuous!”
Again, my question: what is your null hypothesis? Mine is that people is people, what is happening in one group is probably happening in similar groups.
The climate science null hypothesis is that climate scientists are virtuous, and thus have no need for Feynman like integrity.
Steven (19):
I read a ton of magazines because that’s my thing. I would talk more about mag articles if much of the content of so many weren’t behind a paywall. For example, last month Texas Monthly had a nice special issue on immigration–a pet issue of mine–and I wanted to write about it, but nobody would have been able to read the articles. Same goes for SciAm, Discover and Science, (and yes, Harper’s) to name just a few magazines that don’t make their stories available for free online. I would love to talk more about the great stuff in these mags, but it doesn’t seem right if people can’t access them easily online.
Anyway, you write:
he didn’t really show much beyond the ‘soft’ psychological sciences, ecology (multivariate) , and medical research (which, involving human subjects, involves both multivariables AND ethical issues).
You can only fit so many examples into a magazine story and retain the flow. Stories like this aren’t meant to be listy and tick off all the fields that fall prey to this problem. He chronicled a bunch as it was.
@ HD
It’s not accurate in the sense that is VERY incomplete, as indicated. In a figure of speech, I have lifted a corner of the veil for you and simplified the small part of the big picture as much as I could within 2 minutes.
As for my ignorance, feel free to learn something for yourself. Stop by at the library, pick up a paper with model and try to understand what they assume. Mathematically, you won’t make past the first page but you might be able to appreciate the scale of sweeping assumptions that they have to make to get any results.
The macro laws of ideal gas are to the best of my knowledge empirical. Maybe they can be derived from first principles – I don’t know, that’s not my field.
You are right that it doesn’t follow mathematically that large populations cannot be modeled because a single person cannot. Maybe under some other star it is the case. But not here. That’s (so far) an empirical fact.
If I have a bias it’s the opposite of what you think. At some point I was planning a career in macroeconomics.
#21: “But I see that baby-and-bathwater implication (humans are flawed, therefore science can be flawed, therefore climate science is all wrong) is already well represented on the comments here, from the expected sources.”
You’re sounding Bloomian. I didn’t suggest that and wouldn’t.
Of course Sashka is correct – you don’t have to assume rational man outside of economics.
@ lucia
Yes, derived with assumptions but well-justified ones. The choice of stress tensor (we used to call Newton’s law but I forgot why) is well justified on molecular scales. On some level everything in physics is empirical.
climate models don’t solve the Navier Stokes equations.
That’s correct. Climate models solve Reynolds equations http://en.wikipedia.org/wiki/Reynolds-averaged_Navier%E2%80%93Stokes_equations at best or ad-hoc higher order equations (with artificial boundary conditions) at worst.
There simply isn’t enough computational power to permit them to be solved.
That’s correct but irrelevant. We don’t have even the theorem of the existence of the solution: http://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existence_and_smoothness
But computer models that treat the equations exactly exist.
That’s correct but only in the idealized situations AFAIK. These models won’t help modeling climate no matter how much computer power we have.
We don’t derive the navier stokes from a mathematical model of a single atom.
I know. I was just trying to build an accessible analogy: person->economy ~ Navier-Stokes->Climate. I didn’t imply anything about molecules.
Certainly some sciences are more challenged by these problems than others, but all have to deal with them in some way. Climate science is no exception, and that includes the more scientific climate skeptics. None of this means scientists are slaves to groupthink or personal bias, but there are legitimate problems to keep in mind.
Sure, these sorts of arguments can be appropriated to tarnish the authority of science, but they can also be used to make it more robust. Every medical study that makes headlines and is proven to be bogus within a year or two contributes to public skepticism, as are claims about climate that end up easily refuted (I think the problem is much worse in medicine by the way).
We need a healthy skepticism towards scientific claims, and a realistic views about science’s inherent flaws, some of which can be addressed by paying closer attention, but not entirely.
I think the trick is how we can manage with this more skeptical attitude toward science and still recognize when we need to take action – when do we accept that our favorite food might give us cancer, or GHG emissions be cut.
Blind faith in science inevitably leads to error, but skepticism can also be extended indefinitely to prevent us from seeing the obvious.
seeing the obvious
like what?
Can’t believe Huge Difference (#1) still believes the science is settled and Naomi Oreskes has a clue.
“But what relevance does this have to the settled science of climate change and the fact that Naomi Oreskes a science historian, never found a single peer reviewed paper that disagreed with the consensus?”
Her careless study has been thoroughly debunked by Peiser and later Schulte.
Here is a list of 800 skeptic peer reviewed papers
http://www.populartechnology.net/2009/10/peer-reviewed-papers-supporting.html
And many, many here – growing every day
http://nipccreport.org
Positive result publication bias is universal, all sciences. Getting funded and published for re-tests and negative results is a steep hill to climb, and most aren’t interested in attempting it, so there’s a kind of self-pre-selection going on about what experiments or studies to run. Then, the paper hits the editors and they have a limited number of pages available, and prefer not to fill them with tedious confirmations or non-confirmations rather than dramatic original “new” claims.
So in physics or any science, the results that get “replicated” and reported are few, generally only those which are opposed to someone else’s theories. Since possible experiments are many, and funding and publication space is scarce, false results will tend to stand for a long time. The “90% of published physics papers are wrong” may not be so far off. In “climate science”, it’s probably far higher than that.
John (24):
As a former editor, I’m accustomed to having a little leeway with headlines. At any rate, I don’t see it the way you do and either did the New Yorker, apparently, since their subhead read: “Is there something wrong with the scientific method?”
Zajko (31):
One of my favorite parts in The Atlantic article relates to John Ioannidis:
The irony of his having achieved this sort of success by accusing the medical-research community of chasing after success is not lost on him, and he notes that it ought to raise the question of whether he himself might be pumping up his findings. “If I did a study and the results showed that in fact there wasn’t really much bias in research, would I be willing to publish it?” he asks. “That would create a real psychological conflict for me.”
SC (33):
I think HD was being sarcastic there. Speaking of, I’m guessing your blog handle is meant to be funny?
KK: You’re sounding Bloomian. I didn’t suggest that and wouldn’t.
Bloom? As in Bialystok &?
Anyway, I certainly wasn’t suggesting *you* were making that implication, or that you ever would. Nor am I chiding you for reading some of the same magazines that I do…quite the contrary.
Hmmm…Not sure unmanageable climate change is an ‘experiment’ I’d care to replicate. Seems to me that the Precautionary Principle would argue that lesser harm would come from avoiding the experiment altogether.
For Bloomian, see comments 4 & 7. Anyway, glad to hear you read those mags, too. I worry people are not reading magazines as much these days if they can’t be had for free online. So I’m actually quite sympathetic to mags that put some if not all their print content behind a paywall.
Oh, and based the links he/she provided, I’m betting the entity styling itself ‘Senior Climatologist’, isn’t one.
I did enjoy learning that the Latvian Journal of Physics and Technical Science , Irrigation and Drainage , and The Cato Journal can join Energy & Environmental Science and Climate Research as safe havens for ‘skeptical’ climate papers.
Sahska (32)
The obvious could be considered as any great weight of evidence. My point is that skepticism is always an option, and one can always find reason to doubt any claim even if well-supported. And I don’t think AGW is obvious, but even if it was other people might not see it that way. How obvious does something have to be before we act on it? Good question with no easy answer.
KK: yes, editors get choice of titles.
I assume you would not then object to a title:
Journalism is flawed
When the article focused on problems in particular subgroups within journalism, without carefully observing that problems vary and are more likely than some areas than others. One of the articles correctly specified “medical science”.
===
When you say you do not see it that way, are you saying that medical research is the same as physics? Or do you mean something else?
The story at http://www.wired.com/magazine/2009/12/fail_accept_defeat/all/1 was interesting, and had some good points. But some of it seemed to fall into the same over-generalization problem as above, to support a point-of-view. Lehrer obviously didn’t understand the context of Bell Labs-Holmdel, and he started the piece with trivially-checkable credibility-destroying gaffes about Penzias&Wilson:
“were using a radio telescope in suburban New Jersey …And so they had retrofitted an old radio telescope, installing amplifiers and a calibration system to make the signals coming from space just a little bit louder. But they made the scope too sensitive. Whenever Penzias and Wilson aimed their dish at the sky,…”
It wasn’t a radio telescope, it was a horn antenna used earlier for satellite work. It was not a dish.
Google: penzias wilson you get pictures of the antenna
Now, that is not a good start. Likewise, had he understood a bit more about the kind of work done at Bell Labs Holmdel, he might have had to rethink some of his story.
[Note: I worked at Bell Labs for 10 years, worked closely for 2 directors , one of whom was later CTO and the other BTL President. Arno Penzias is a friend of ours, whose stories over dinner are always amusing. I would not over-generalize from Bell, which was its own unusual place, with 25,000 people in R&D. But I’ve visited hundreds of R&D places…]
Despite the errors, some of Lehrer’s story was reasonable, but some of it seemed like someone not very close to science with a story in mind, using ideas whose applicability is mixed.
“While the scientific process is typically seen as a lonely pursuit “” researchers solve problems by themselves “” Dunbar found that most new scientific ideas emerged from lab meetings”
SO, *who* sees it this it way? We didn’t see see it that way in Bell Labs.
In the 1990s, I was in hundreds of meetings with a big variety of science groups (physics, chemistry, biochemistry (pharma), climate science, etc .. spread across university, government, and corporate). *They* didn’t see it that way. There were lots of teams, often interdisciplinary.
A friend of mine is President of Stanford. *He* certainly doesn’t see it this way and works very hard to stir up interdiscplinary teams.
I’ve spoken at hundreds of universities, and this “lone researcher” thing certainly happens, but I’ve been in too many national labs and companies that try very hard to make sure it’s easy for people to collaborate . People design buildings to help that happen.
“Modern science is populated by expert insiders, schooled in narrow disciplines.”
Maybe, but I know quite a few people who break that rule. Interdiscplinary work is often where the action is.
I know pithy over-generalizations make for punchy stories, although avoidance of trivially-checkable errors would be nice, also. (I don’t know where Dicke was planning to build, but Princeton is ~due West of Holmdel.)
Many of the commenters were impressed, but I have to wonder how many of them had spent a lot of time in or around a variety of world-class science organizations.
And I do wish people would distinguish between the human issues common to all, and the distinctly different kinds of problems prevalent in different disciplines.
John Mashey (41):
I wonder if your question is prompted by this tweet, which I saw last night. I find it striking that some climate scientists and climate observers would get prickly over my headline, post, or the Lehrer’s New Yorker article. I think this reflexive sensitivity is unwarranted but perhaps understandable given the events of the last year.
Anyway, do I think journalism is flawed, too? Absolutely. Don’t we see that on a regular basis? And like science, journalism engages in a continuously corrective process.
Keith, half of your blogroll is about climate change, THE major tag of yours is “climate change”, way more than half your tags are climate change related. You’ve got one tag on swine flu, one on anthropology, and that’s about the only science related tags outside of climate that I saw on the quick glance. So why, oh why, would people think this blog post and its headline is about climate science?
Why should it not be about climate change? If not explicit, quite clearly those of a sensitive nature are distinctly inferring reference to climate science. But sometimes, as Freud would say, sometimes a cigar is just a cigar.
Cats smell their own muck first.
#43:
Do I write often about climate change related issues? Sure. Do I think my post has relevance to climate science? Sure.
But my tag on the post is “science.” If I wanted to make an explicit connection, I would have done so. I still may follow up with a post of that nature, time permitting. If I did that, it would probably go something like this:
In The Atlantic piece, the reax to John Ioannidis is one of relief and his criticisms are welcomed by his colleagues.
Contrast that reception to the one Judith Curry got last year around this time, from her colleagues, when she was talking about “tribalism,” “circling the wagons,” etc. Pretty hostile, on the whole, wouldn’t you say?
So I would say, based on the anecdotal reax to this post that I’ve seen thus far, it would seem some climate scientists continue to exhibit a reflexive defensiveness– even when critiques of science are not explicitly directed at them.
@39 Steven Sullivan. Can we therefore conclude that you believe where something is written is more important than whether or not it is correct?
Concerning you #19. Yes, there is the baby bathwater possibility and that must be guarded against. However, can you see the other side of the coin? If the same situation has been found across a number of sciences then the only reasonable conclusion is that it is present in climate science as well. Any other conclusion must be based on some sort of “special pleading” by assuming that is something that “somebody else did”.
I for one can see some paralells between medicine and climate science. We could start with the very large number of ill defined (as to absolute value) variables.
All that Ioannidis proves is that scientists are human beings (who woulda thunk it?) and the peer review process isn’t quite what we all hoped it would be for reversing incorrect findings.
@ 41 John Mashey. He was writing for Wired so I think it’s a bit hair splitting arguing Horn or Dish. I do realise that correct nomenclature is required in “in field” discussions, but to the general public, it’s a Dish.
As to *who* sees science research that way, probably most of the general public. I would suspect that most people think of scientists (of all stripes) as guys in white coats working in a lab or large cubicle with maybe 3 coworkers. Dunbar was pointing out that the general perception is wrong.
He was telling them about the group dynamics that you are talking about. IOW, he agrees with you.
@ KK (42)
I find it striking that some climate scientists and climate observers would get prickly over my headline, post, or the Lehrer’s New Yorker article.
You must be kidding.
Suspicion always haunts the guilty mind;
The thief doth fear each bush an officer.
Henry VI, part 3, Act 5, Scene 6
A PhD does NOT an Einstein make. There are sooooo many PhD’s awarded every year, it must have something to do with the size of the human footprint on the planet, right? Just because Harvard Jones says something doesn’t mean it’s so. Same with Princeton Smith or Wisconsin Schmidt, or Chicago Smythe, ad etc. The point is that there are very few folks at the TOP of their field.
The J. Q. Public types are also very educated. They instinctively know (due in part to their exceptional backgrounds but also to something called Common Sense) that a PhD does NOT an Einstein make.
“Publish or Perish” seems to lead us to a point in time when 99.9% of all that is published is worthless speculation. How did we get here?
@ Rob Watson (38)
“Not sure unmanageable climate change is an “˜experiment’ I’d care to replicate. ”
Perhaps this is o/t, but Rob, I don’t think we have any choice in the matter. Climate change is and will remain unmanageable. There’s nothing we can do about it. The great natural experiment will continue regardless.
We can and do influence climate, at least locally, mainly by altering our local environments. But whatever we do, climate change is, sooner or later, coming our way.
And I think it’s a great idea for us to economise on the wasteful use of fossil fuels. But I’m not such a fool as to imagine that using less of fossil fuels will stop climate change. It can’t and won’t.
Keith, when I read a blog post, I usually do not look at the tags, at all, especially if the tags are colored in an eye-piercing white on light gray. Just as I do not check on a single user blog, whether or not that single user signed the post. You could have caught me with a guest post by someone else, as well. I use tags only as a means to find archived posts arranged by general theme. Maybe that’s just me.
So I would have preferred you to state up front, whether or not this post is meant to be read in the context of climate science, the main subject of your blog; especially so, as allegations of flawed or fraudulent science are commonplace for skeptics-in-name-only in the field of climate science. When discussing the content of a post, I consider the tags to be the equivalent of legal fine print (‘Hey, look, I had the humor tag included. It’s not my fault, that you could not tell it’s satire from the text.’). Again, maybe that’s just me.
Sashka
Yes, derived with assumptions but well-justified ones.
But you said “first principles” which is misleading. I’m all for empiricism in science, but your calling it “first principles” is bound to mislead those who are unfamiliar with the NS. Those familiar will recognize you as an exaggerator.
That’s correct. Climate models solve Reynolds equations
Of course I’m correct. And my broader point is your omitting this information is misleading in context of your discussion to suggest we can somehow know that climate models are better than economic models. The introduction of ad hoc parameterizations into the Reynolds equations could at least potentially make climate worse than economic models.
That’s correct but irrelevant. We don’t have even the theorem of the existence of the solution
In context of practical computational solutions of the NS, and climate models, it’s your theorem or existence that is irrelevant.
These models won’t help modeling climate no matter how much computer power we have.
Huh? Strictly speaking, DNS isn’t theoretically limited to simple geometries. It is only applied to simple geometries owing to lack of computer power in any real situation!
I know. I was just trying to build an accessible analogy: person->economy ~ Navier-Stokes->Climate. I didn’t imply anything about molecules.
But why build an “accessible” analogy that is so wretched? Given the words in your comment, your “NS->Climate” model analogy really goes:
something very small we understand on first principles -> first principle derivation (false) -> NS -> left out massaging into Reynold’s equations with parameterizations ( which is known to introduce errors in every field where it is used and we even know why) -> climate models.
But you want to distort and present that as
NS->Climate Models so you can contrasting the real change with
person->economy
And based on your highly selective discussion of the relation of climate to the NS you suggest we must conclude climate models must be better.
Up thread, someone wrote “Selective reporting is a form of cheating.”. It is cheating. So maybe you ought not to do it?
lucia
Not only it was selective reporting I (not you) was the first one to say so. I was omitting information on purpose and I was very explicit about it. What’s your problem.
I agree that instead of first principles I should have said from Boltzmann equation which won’t make any difference to most.
DNS had never been demonstrated to work in a general case. For all we know the solution will break down in some way, most likely via computational instability, over a finite (probably short) period of time. The existence theorem would be very helpful for someone trying to claim otherwise.
I’m not going to defend the this analogy. Any analogy is imperfect. If you find it wretched it’s fine by me. Feel free to suggest better one. If you disagree with my point in general and feel that economic models are better it’s fine, too. I’m interested to listen to your arguments.
Of course I’m correct.
I like that attitude. You probably think that you know more about climate modeling than I? That’s pretty funny. You want to duke it out? BTW, why are we having this hostility?
[…] Now, the profession of science, that for starters has a fundamental problem with social skills, is not only being laughed at — it’s now being chopped at with hatchets. Keith Kloor started assembling the pieces of this science public image disaster yesterday with a discussion on his blog Collide-a-scape. […]
KK: ” And like science, journalism engages in a continuously corrective process.”
Except that in science it works, but in journalism it doesn’t. Individual errors do sometimes get corrected, but the same ones keep getting repeated. IMHO the conflict narrative would have to be abandoned for that to change, along with the dominance of for-profit entities.
Interestingly Lehrer was on the *Brian* Lehrer NPR show ton WNYC this morning, and made a point of specifically saying his article doesn’t mean we should reject the science behind evolution and global warming… two examples I expect doubt picked because of the decades of persistent and often unwarranted ‘skepticism’ they’ve encountered from the public.
here’s a link to the show:
http://www.wnyc.org/shows/bl/2010/dec/10/studies-show-or-do-they/
Keith, Ionnadis and Curry is not really a great analogy. Curry’s critiques have been rather less disciplined (or consistent) than Ionnadis’. Look, I realize she makes a great story, but you gotta watch for the ‘wow’ factor there too.
@ Steven (53)
I don’t think it’s his call to decide where the article applies. His job was to open the lid and let the genie out (let’s pretend that he wasn’t yet). Hat tip for that but we’ll take over from this point on.
I submitted this trolly thread on FARK, I admit it:
http://www.fark.com/cgi/comments.pl?IDLink=5814522
FARK has many dimwits but actually a large population of very talented, smart, and knowledgeable individuals from many backgrounds, climate science included.
Note how many true believers there jump right into John Ioannidis’ trap and many of the other traps discussed in this thread.
Scientist-turned filmaker Randy Olson posted a related essay today on his site.
Steven (55):
Thanks for the link. I agree with Lehrer.
NYTimes today:
http://www.nytimes.com/2010/12/10/science/10anthropology.html
Anthropologists have been thrown into turmoil about the nature and future of their profession after a decision by the American Anthropological Association at its recent annual meeting to strip the word “science” from a statement of its long-range plan.
The decision has reopened a long-simmering tension between researchers in science-based anthropological disciplines “” including archaeologists, physical anthropologists and some cultural anthropologists “” and members of the profession who study race, ethnicity and gender and see themselves as advocates for native peoples or human rights.
During the last 10 years the two factions have been through a phase of bitter tribal warfare after the more politically active group attacked work on the Yanomamo people of Venezuela and Brazil by Napoleon Chagnon, a science-oriented anthropologist, and James Neel, a medical geneticist who died in 2000. With the wounds of this conflict still fresh, many science-based anthropologists were dismayed to learn last month that the long-range plan of the association would no longer be to advance anthropology as a science but rather to focus on “public understanding.”
Until now, the association’s long-range plan was “to advance anthropology as the science that studies humankind in all its aspects.” The executive board revised this last month to say, “The purposes of the association shall be to advance public understanding of humankind in all its aspects.” This is followed by a list of anthropological subdisciplines that includes political research.
The word “science” has been excised from two other places in the revised statement.
…
Dr. Peregrine, who is at Lawrence University in Wisconsin, said in an interview that the dropping of the references to science “just blows the top off” the tensions between the two factions. “Even if the board goes back to the old wording, the cat’s out of the bag and is running around clawing up the furniture,” he said.
He attributed what he viewed as an attack on science to two influences within anthropology. One is that of so-called critical anthropologists, who see anthropology as an arm of colonialism and therefore something that should be done away with. The other is the postmodernist critique of the authority of science. “Much of this is like creationism in that it is based on the rejection of rational argument and thought,” he said.
One way to see this is that science is under attack from outside forces like Big Tobacco that just want to force babies to smoke cigarettes.
Another way to see this it that science is under attack from forces inside academia that want to coopt science to shield their politically correct politicized political agendas.
The appropriate response in either case is to just say, “No Thank You”, stop calling people you disagree with deniers, and start listening and acting like grownups.
Scientists are people, no better no worse, not more virtuous, not less virtuous, and living in a democracy, not a technocracy, and they should hope, not in a politically correct, politically politicized dictatorship of the political activists.
#48: … “Publish or Perish” seems to lead us to a point in time when 99.9% of all that is published is worthless speculation. How did we get here?….
67.3% of all statistics are made up. 🙂
Sashka
Of course I’m correct.
I like that attitude. You probably think that you know more about climate modeling than I?
I have no idea if I know more about climate modeling than you. But I do know material from senior year fluid mechanics courses. You made mis-stements. I corrected them. You tell me I am correct. Well, of course I am correct on those because I posted well known, indisputable information about the navier stokes equations. I have a ph.d. in mechanical engineering, my degree was in multiphase flow, and yes, I know that what you said about the Navier stokes and modeling is simply…well… wrong.
I was omitting information on purpose and I was very explicit about it. What’s your problem.
It appears your purpose in omitting information from your discussion of the connection between fluid mechanics was to mislead people as to the degree to which climate models are “rigorous”.
I Liked the interview on NRP and the Atlantic article, but was not too impressed with the Wired article. I don’t think the author really knows what it is like to do science. When one gets unusual an unexpected results it is usually because one made a mistake. Most scientists have a lot of experience have unexpected results and when they track down the source they find an error. It is not just that the results don’t fit into ones understanding, it is that one has a lot of experience being wrong. When something unusual comes up you need a lot of careful checking to make sure it is repeatable and that you didn’t screw something up. This takes a lot of work.
And there is a big difference between medicine and physics. Medicine is much much sloppier and less certain then most physics. The culture and funding is very very different. 15 years ago when my wife had cancer, I started reading the medical research and I was amazed that just about all the treatment protocols was based on really poor statistics.
Charlie Petit at Knight Science Journalism Tracker takes up the New Yorker piece.
Mr. Kloor –
The phenomenon described in these articles is nothing new to any honest person who has worked in science. It is telling that the Steve Blooms, Gavin Schmidts, and Mike Manns of the world go into attack mode when the topic is even discussed. What we skeptics realize is that the current political, umm, climate, is very fertile soil for confirmation bias. With saving the world, punishing big corporations, redistributing income from first to third world, creating egalitarian lifestyles, etc – how many university types can resist fudging a number here and there, or “hiding the decline”. And fudging a number here and there is all that is required when the subject – like much of medical and pharmaceuticl research – involves finding the very small signal in the very large noise.
Tom C:
Just to be clear: to my knowledge, Mike Mann and Gavin Schmidt have not attacked any of the articles I mentioned in the post.
Additionally, I think climate skeptics would do well to consider if they are being unduly influenced by any of their own biases, as well.
Fair enough.
I think Judith Curry’s response to Congress’ additional questions is very relevant to this thread:
http://judithcurry.com/2010/12/09/testimony-followup-part-ii/#comment-20664
(1b) “The principal climate data records should be maintained by government agencies, with full documentation, quality and version control, complete documentation, and support to respond to user queries. University research groups are ill equipped to handle this, and researchers generally find the painstaking work of quality control to be scientifically boring.”
I think every body here could agree with this bit as it would benefit everyone, and ‘ahem’, was a suggestio I made at C a S many months ago, ie free up the scientists/researcher time.. anyone could just use it, without having to maintain it.
I wonder it it will happen
Two brief contributions to the discussion.
First, I think it is important to distinguish between situations where the ‘flaw’ (in quotes because I’m not convinced we ever know the truth) is internally generated by slavishly following an accepted research protocol (e.g. significance chasing) and situations where it involves external considerations (e.g., medical research funded by drug companies that downplays the significance of potential side effects or research in areas where the findings have been politicized). I’m not suggesting there is clear separation between the two or that it is ever possible to separate scientific research from cultural context, but I think there are multiple processes going on here and maintaining the analytical distinction is useful in teasing them out.
Second, another way to view these issues is through the lens of Steve Shapin’s arguments about the literary quality of science. Back in the 1600’s, scientific knowledge was transmitted by demonstration. Scientists would get together and show one another particular physical phenomenon in person. Seeing was, quite literally, the basis for believing. One of the major accomplishments of the scientific revolution, according to Shapin, was the process of ‘virtual witnessing’ — that is the development of various conventions about how to convey the results of research through writing such that readers believe that they have witnessed the demonstration. It is those same literary conventions which are now being attacked as the source of error.
Gary, thanks for stopping by and offering these two perceptive observations.
On a separate note, I’d really like to see you (or one of your colleagues) weigh in at Ecological Sociology on the big anthropology dust-up.
For Huge Difference:
You stated….But what relevance does this have to the settled science of climate change and the fact that Naomi Oreskes a science historian, never found a single peer reviewed paper that disagreed with the consensus?
You just made an incorrect statement. Climate science or any science is never settled by a consensus. Warmists like to use that term. There is no empirical proof of global warming. If you can find any proof, please share it in this forum. As far as Orestes is concerned, you just proved the tenets of this article. She searched and found the result she was looking for. BTW, Orestes was recently challenged to debate this issue and turned it down. HMMM???
I’m sorry Keith, I see I need to pepper my comments with smileys.
Lucia #23:
We have no closed form solution for the Navier-Stokes. But computer models that treat the equations exactly exist. Look up “direct numerical simulation”. Climate Models don’t solve them because the flow field is too large.
I can’t find a sense where that is really true. General Circulation Models of the climate system (GCMs) are indeed “primitive equation” models, wherein the NS system is explicit and plays a fundamental role. Like other computer models of fluids, small scale phenomena cannot be resolved. I do not think this trears teh equations “exactly”.
It is true that the workarounds for this problem are far weaker in GCMs than in other fluid models. That’s a big part of the research problem. But that’s not really at issue in the assertion.
I want to say “thank you” for this post. I started reading related information on this general subject about 7 years ago. The information isn’t necessarily easy to find. There aren’t a lot books (or magazines) that deal with bias in science. A few that deal with fraud or related activities, but not too many of those either. One of my favorites is “Voodoo Science” by Robert Park.
I won’t touch the “bias in the media” thing. Don’t need to – it’s obvious to the most casual observer. One of the reasons I started coming here some years ago was that your bias is far less pronounced and much more even handed than most. Thank you for that.
I spent 40+ years dealing with scientists – most of them atmospheric physicists – and this post (and the related links) hold no surprises for me. Not that the people I worked with were dishonest, but they DID have their own biases – some unconscious, some due to funding issues, some due to publishing issues, some due to personal antagonisms, some due to hardheaded belief in their own rightness – and a VERY SMALL minority due to personal ethics issues. Someone’s gonna want details – there’s always one of those. My message for them is “fuggetit”. Except for the last category, I liked and respected those people. But I never, NEVER bought into the illusion that they were perfect or unbiased or incapable of error. Because they wren’t. A part of my job was to do what I could to keep them from making avoidable errors. It wasn’t always easy and I didn’t always succeed. 😉
Reading the links, in particular the Atlantic piece, was like a walk through my past – different science, different faces, same situations. To assume that the same words/concepts can’t be applied to climate science is a measure of the blindness and bias of those who so believe. That’s not a question.
Yeah, I’m back. Hiking doesn’t work well when the bionic knee starts doing weird things. So the knee mechanic is gonna fix it in Jan. We hope.
Ref – Huge Difference Says:
December 10th, 2010 at 5:58 pm
NYTimes today:
http://www.nytimes.com/2010/12/10/science/10anthropology.html
Anthropologists have been thrown into turmoil about the nature and future of their profession after a decision by the American Anthropological Association at its recent annual meeting to strip the word “science” from a statement of its long-range plan.
________________
The dominoes have been falling for sometime now, looks like they’re starting with A and going to Z. When everyone wants a PhD in NucPhys what else can you do? Professionals in the Field really do need a bigger say in who’s in charge of Colleges and Departments in the Academic World, but what can you do when the “professional organizations, associations, and tea-parties” are so often worthless front offices run by PoliSci majors? Not to worry! Things like this have been going on for years now in Elementary and Primary Education –I even understand that NYC teachers CAN’T be fired now. (Sarc and Crying Off)
Ref – Nosmo Says:
December 10th, 2010 at 6:34 pm
#48: “¦ “Publish or Perish” seems to lead us to a point in time when 99.9% of all that is published is worthless speculation. How did we get here?”¦.
67.3% of all statistics are made up. 🙂
_________________
True;-)
Some good recap of online reactions to the Anthro controversy:
http://blogs.plos.org/neuroanthropology/2010/12/11/anthropology-after-the-science-controversy-were-moving-ahead/
It appears your purpose in omitting information from your discussion of the connection between fluid mechanics was to mislead people as to the degree to which climate models are “rigorous”.
You misunderstood completely. I didn’t say or imply anything about climate models being rigorous. If you care to look up the stuff I usually write it is more often than not about models being wrong. My point was that the climate models (unlike economics) at least start with something that we consider well established – Navier-Stokes. Your nitpicks about how rigorously N-S is derived are technically correct but irrelevant to the point. That being: if you can start with a good micro-model (climate) then you can end up with a good macro model or a bad one. If you start with a wrong micro model (economics) then you are doomed.
Keith,
Thanks for the plug for our blog and the gentle nudge to comment on the situation in anthropology. I’ll try and do so in the next couple days.
Gary
KK
1) I don’t see tweets.
2) Oddly, my issue is more with journalism practices regarding the Penzias/Wilson story, because it had a strong feel of someone with a story idea finding a part of story that fit, and not bothering with much fact checking or looking into context. Others may be impressed by a shallow story one could get from Wikipedia. I’m afraid I am not, because it wouldn’t have taken much work to have made that part of the story more accurate and actually more interesting.
This is not to disagree with the general issue, but while the bare bones of the Penzias/Wilson story is famous, easily gettable from Wikipedia, Lehrer didn’t talk to Arno or dig any deeper. To be blunt, he didn’t show the slightest understanding of how Bell Labs research worked or the signal/noise issue.
Arno was in Bell Labs Area 10 (“research”), split between Murray Hill and Holmdel, it was typically ~7% of Bell Labs (which when I left , was about 25,000 people). Think of it as organized serendipity, run that way since 1925. They hired the smartest people they could get, within general areas of interest (like radio), who pretty much worked on what they felt like much of the time. They were the R1 and some R2 in <a
href=”http://dotearth.blogs.nytimes.com/2008/12/12/r2-d2-and-other-lessons-from-bell-labs/”>R2-D2 and other lessons from Bell Labs</a>.
We knew they would often produce breakthroughs, but the mantra was “never schedule breakthroughs.” The proximity of numerous researchers guaranteed random interdiscplinary encounters; such were sometimes were purposefully encouraged in various ways. The Bell Labs library system ran a terrific information dissemination system, to make sure different researchers easily could know what was going on. Internal seminars were very strong, and it was normal to expect you might need to seek help across disciplines.
[I make no claim this is universally true, and in fact, in many university, the “stovepipe” effect is strong, departments are individual fiefdoms, and higher administration sometimes despairs of fixing this.]
BTL Executive management fiercely protected the ability of Area 10 to follow serendipity around (after all, it’s kind of hard to explain why the phone company had people using a leftover horn antenna to look at the universe rather than at satellites. It would be hard to explain why the guy who created UNIX later spent ~5 years or so building a chess computer that was world computer chess champ for years.)
While one could not *make* Area 10 folks be interested in a problem, or do anything that looked like scheduled development, quite often, people in development divisions would take problems over there, and they might be able to solve them, or it would suggest new lines of research.
More often, people in development divisions would maintain close contact with people in the correspond parts of Area 10, visit regularly and see what was new. (I had a history of doing this, but was also explicitly told by my bosses to make sure I kept doing it, i.e., this behavior was institutionalized.)
The point of all this is that it is actually possible to fight the problems described in the article, but doing so requires key management practices. Well, monopoly money helps. note of course, that serendipity harvesting was already there when Arno arrived, but he also went on the *run* Area 10, and I know remembers the value of serendipity.
Bell labs was extreme case, but other top R&D places understand this also. 3M is fairly famous for it, with PostIt notes as a famous case.
The other issue is perhaps a little more esoteric, but Lehrer didn’t mention it either. I’d guess he didn’t know about it, but it wouldn’t have fit his general idea very well, either.
Unusual data may or may not mean something in any discipline. But Bell Labs Area 40 was “Transmission”, based mostly in Holmdel (where Arno was) and Whippany, where I was, in fact, I worked 5 years in a lab that was part of this. This usually included copper, radio, satellites, undersea cables, later fiber optics, and the radio part was the development lab correspondng to Arno’s research lab.
People build transmitters, (possibly transmission medium), receiver. They send a signal, it acquires noise, it gets to receiver. They spend huge amounts of energy trying to identify cause of noise and engineer to lessen it. it is completely maladaptive for such people to think that some noise is an amazing new effect that will lead to a Nobel. Almost always, it is a design problem, or just irreducible noise, in which case one engineers the systems to expect that amount of noise, degrade the expected bandwidth and get on with it.
Of course, when you are receiving a signal you didn’t send, you get signal+noise, and you have to disentangle which is which and often is not easy. The point of all this, is that if there a single area of research that would (and should) treat odd data as noise to be gotten rid of, it is this one, because in fact, it almost always *is* noise.
SO, I end up with several issues:
a) Simple fact-checking is nice.
b) Crediting people with exposing an issue may be OK, but it is really weird to pick an example from, of all places, an organization that had a 50-year history of provable understanding of this issue and working hard to manage it, and one of the all-time great records in research and innovation. I am sorry, I am as unimpressed as I am when I see stuff from the Breakthrough Institute about R&D breakthroughs, written people who for the most part haven’t been involved in serious R&D efforts. Again, this is not to argue with the general idea that research isn’t perfect … but that isn’t exactly news.
c) The other thing is the overgeneralization from medical science to science as a whole, using that specific paper as an example. That is just not very helpful, except to people who wish to cast doubt on science as a whole. Sadly, they seem to be succeeding in the US, which I’m afraid will likely lead to China being the center of science in 50 years or so. Lots of people learned well from the cigarette tactics.
The irony of watching the AGW believer community turn into out and out deniers is rich indeed.
Pointing out where bureaucratized science is bogging down is hardly anti-science, by the way.
Keith,
My comments on the anthropology controversy are at http://ecologicalsociology.blogspot.com/2010/12/much-ado-about-not-much-in-anthropology.html
Gary
Gary,
You don’t disappoint. Really interesting take and I’d like to discuss it, but too busy today. Check back tomorrow morning for a post.
The New Yorker piece has triggered lots of good commentary in the science blogosphere. I’ve provided links to all in an update in the body of my post.
Scientific Denialism?!? Too funny. The ‘theory of climate change’ says a lot of different things, many of which are poorly supported and hotly contested. Does the author feel that his comments about testing and robustness apply to:
Climate sensitivity to CO2, negative vs. positive feedback, catastrophic projections, the accuracy of climate models?
“In poll after poll, Americans have dismissed two of the most robust and widely tested theories of modern science: evolution by natural selection and climate change. These are theories that have been verified in thousands of different ways by thousands of different scientists working in many different fields.”
This is real disappointing. To compare the evidence of natural selection is on par with AGW seems ridiculous. There have been 1000’s of tests available to disprove natural selection and it has withstood over 100 years of those tests. AGW theory on the other hand is a recent phenomena. The first real first test of AGW was “increases in CO2 will cause warming”. This test has failed badly over the last 10 years or so. Then after the fact “scientists” try to justify the discrepancy by a variability argument. The variability argument was not offered before the fact. This is precisely the problem with the article points to. I just don’t see AGW scientist offering tests that would disprove their theory. They can’t offer tests because we have seen there is a problem with amount and quality of data is poor. Then comes Climategate, which deals with almost every problem presented by the article. There is an irony here not missed by those who truly believe in the scientific method.
Have we all forgoten Alan Sokal? He proved peer review is worthless. Who you are determines how tight the review process is on any given paper. Climate science is a good example of this.
87:
Social Text. I remember it well, but that was a different kettle of fish.
Peer review is worthless? Now that’s what I call throwing out the baby with the bathwater.
“”In poll after poll, Americans have dismissed two of the most robust and widely tested theories of modern science: evolution by natural selection and climate change. These are theories that have been verified in thousands of different ways by thousands of different scientists working in many different fields.”
Mr. Kloor:
Good Grief! I liked the writeup until I saw this unbelievably ironic statement which essentially goes against all the wisdom of those cited in your post! You must really want this “climate change” thingy to be true, eh?
When “climate scientists” can explain what caused the Roman Warm Period, Dark Ages Cold Period, Medieval Warm Period, and Little Ice Age, I may “listen” to them. It would be nice if they could also explain the very poor correlation between anything man has done and the widely varying “climate changes,” whatever the hell those are in your mind.
Just because Lehrer and KK think the climate science community is immune from the bias phenomon, does not make it so. Climategate showed that clear enough. CO2 driven anthoprogenic climate change has NOT been verified in any way shape or form. Many papers have climate change as an explicite presumption/assumption, via GCMs or what have you. All climate change has is a theory and a tenuous co-relation between global temperatures with rising atmospheric CO2. That being said, I believe rising CO2 levels have an effect, but the effect has not been separated from natural variations to a signficant degree to warrant the Catastrophic AWG hype that is out there, eg sea level rise and the poor polar bears.
#90: “Just because Lehrer and KK think the climate science community is immune from the bias phenomon, does not make it so.”
Where does Lehrer say that? Where do I say it?
“…This phenomenon doesn’t yet have an official name,…”
Oh, yes it does. The name is simply not used in polite conversation.
I would agree that many studies have documented climate change, which your readers should understand is entirely different from human induced global warming. While science has shown climate change to be very common, proof of the contribution of humans to this change is very much in debate and we should not make the very same mistakes described in this paper of wanting to prove something so badly that we wrestle with the data until it “proves” us right one way or the other.
Keith,
Thanks for the updates. Lehrer’s response in Wired is excellent, and esp relevant are the first question (that you partly quoted) and the last (about the context being important to test the strenght of a claim).
Fred N.,
Lehrer’s argument as to why evolution and cliamte change are robust scientific theories seems to be more along the lines of the multiple independent lines of evidence, accummulated by many individuals from many different disciplines over a long period of time.
Contrast that with his example of a theory put forward by one person which in the following few years is found to be less important than the author initially claimed.
A totally different ballgame alltogether.
Also read Lehrer’s response: It’s the context around a claim which signifies its strength. The multitude of independent studies coming to approximately the same result.
Bart beat me to this, but that was going to be my response to folks like Fred N: it’s the multiple lines of evidence that is key. Skeptics of AGW conveniently gloss over this, or ignore it.
KK, we don’t gloss over them. When we ask we get told about polar bears and shifting migratory habits. While these are evidence of Climate change, they are not evidence that man is the cause.
The normal response to asking to see some of this evidence from multiple lines is handwaving or the bibliography of AR4.
I like to think I’m a reasonable person (my own bias is showing here 🙂 ) so I’m asking. Could either you or Bart show me some of this evidence from many lines that shows man is the cause?
We know the climate is changing, it always has and always will so there is plenty of impirical evidence of that. Show me the impirical evidence that man is the cause.
I’m quite willing to change my mind on CC, but so far nobody has fronted anything strong enough to do so.
@ 80. John Mashey.
That was a fascinating post about the Bell labs. Very interesting.
JohnB,
Tom Yulsman put together this nice overview summarizing the multiple lines of evidence last year.
For over 200 years Isaac Newton’s theories were peer-reviewed “with multiple lines of evidence”. His theories were some of the most tested and highly verified of all time. In 1905, a patent clerk by the name of Albert Einstein comes along challenges Newton with his “Special Theory of Relativity” and challenges the conventional wisdom at the time. Einstein has a problem even getting a job as a professor at any university. Producing theories challenging Newton was difficult and hard to get published at the time and probably hurt his chances of getting a professorship. Similarly in 1915, Einstein produces probably the greatest theory in all of science, “The General Theory of Relativity”. In fact, the science community in the early 1920’s is still reluctant to admit that Einstein could be correct despite evidence to show otherwise and they never officially give him the Nobel Prize in Physics for either theory! Some members of the “old guard” never understood or believed Einstein had produced better theories than Newton. This is the most extreme case I can think of where tribalism, group think, etc.. influenced what people thought was correct. At the time, the “old guard” had 200 years of peer-reviewed articles and evidence to justify their claim!
Now comes along AGW theory with a relatively short history that people believe in just because it has more peer-reviewed articles! And we have written accounts from the Climategate emails that the so called leaders in the field tried to prevent the publishing of anything that went against their theory. Yet some of you still hang on to this notion that “peer-review” is the end all, especially in AGW theory. This type of thing happens over and over again in science. Some of you believe that scientists are so “pure” and without bias that they can’t be challenged. Even the great Isaac Newton was wrong yet you believe there is no way that scientists publishing over a very short time frame getting tons of government funding to “prove” a theory that is politically expedient is beyond reproach? When the leaders in the AGW field are blocking opposing views, I smell a rat.
“Those who cannot remember the past are condemned to repeat it.”
One little fact about Newton is that he regarded himself as a failure. Above all else he was an alchemist. He wrote more on the subject than any other field of study.
He thought himself a failure because of his inability to create a philosophers stone. It is clear today that his problem was he engaged in too much science. Had he lived in the present he simply could have falsified some research and/or published some wild speculation in a major peer reviewed journal and considered himself a success.
John B,
There are some good resources on where the A in AGW comes from at skeptical science, eg
http://www.skepticalscience.com/docs/Guide_to_Skepticism.pdf
http://www.skepticalscience.com/its-not-us.htm
It hinges on a few main pillars, each of which is again supported by a variety of studies from different people over extended periods of time:
– Basic physics
– Paleoclimate shows a certain sensivity to climate forcing in general, and the important role of CO2.
– Physics based modeling is broadly consistent with paleo evidence and with instrumental record, including such varied periods and mechanisms such as ice ages, volcanoes, current warming, PETM, etc.
– Certain fingerprints such as straospheric cooling, nights warming more than days
– Empirical evidence for an enhanced greenhouse effect, both from space (less heat radiation escaping) and from ground based measurements (more heat radiation returned)
This paints a consistent picture of why this is happening, which has only stengthened over time.
Ridium,
Are you also aware that Einstein’s theories were heavily opposed at the time:
When people don’t like what science tells them, they resort to conspiracy theories, mud-slinging and plausible pseudoscience ““ as Einstein discovered
“THIS world is a strange madhouse,” remarked Albert Einstein in 1920 in a letter to his close friend, the mathematician Marcel Grossmann. “Every coachman and every waiter is debating whether relativity theory is correct. Belief in this matter depends on political affiliation.”
@Keith #99 –
From Yulsman’s article –
In 1895, Arrhenius calculated that a doubling of carbon dioxide in the atmosphere would cause the Earth’s average temperature to rise by 5 to 6 degrees C, and the Arctic regions to warm by an even greater amount. A bit later, he suggested that increased burning of fossil fuels could warm the planet so much that it might even avert the coming of the next ice age.
That’s it??? That’s all he has to say about that subject?
Sorry but that’s ignorance – at best. IIRC Arrenhius and others later determined that a doubling of CO2 would result in about 1.5 degC of warming. He also was of the opinion that the warming would be a good thing for humankind. How the hell did those little tidbits get lost? Maybe because they doesn’t fit the AGW dogma?
So… where does Yulsman talk about the logarithmic effect of CO2 warming? When does he get into REAL CO2 physical chemistry?
There’s more, MUCH more, that’s just so wrong with Yulsmans post, but that one kinda pissed me off because it’s exactly the kind of “flawed science” that your original post was about.
And I didn’t even get into the Modeling paragraph or the questionable quality of the global data sets where I have some real expertise. We’ll undoubtedly get to those another time.
You might also note that Yulsman’s take on the effect of the Climategate emails was a massive underestimation.
“…it’s the multiple lines of evidence that is key. Skeptics of AGW conveniently gloss over this, or ignore it.”
This certainly hasn’t been my experience. Sceptics seem very keen to turn over every stone.
@Bart Verheggen
I’m not sure what your point is. You are essentially repeating what I said and proving my point. Even arguably the greatest scientist of all time, Einstein, was apparently wrong about quantum mechanics, a natural extension of his earlier theories. And he continued to question, downplay, and mock the scientists that came after him that developed quantum mechanics field. My point is that no scientist is beyond reproach. Science is messy because it involves people and human behavior and people have biases. My other point is having a theory that has the majority of peer-reviewed research doesn’t really have the meaning people try to associate with it, especially when the “old guard” is preventing research that helps to disprove their theories.
From your quote: “Belief in this matter depends on political affiliation.” Exactly! Polls show that most scientists in American are Democrats. See: http://www.slate.com/id/2277104/ It is hardly arguable that many Democrats favor big government solutions and distrust the free market. This just happens to fit exactly the policy conclusions of AGW theory. It seems to me an objective scientist would admit these biases upfront rather than trying to prevent others from publishing research that contradicts theirs, hide their methods, and massage the data to fit their theories. AGW scientists don’t produce experiments that can disprove their theory. This is the heart of science. It’s the same type of problem String Theory is running up against. You end of getting scientists claiming any kind of event proves AGW. If it warms it proves their theory. If we have bad winter it proves their theory. Hurricanes prove their theory. No hurricanes proves their theory. Most of these are after the fact claims trying to persuade the public but this is not science! All this shows is that weather is variable, which we already know. But that can’t be proof of AGW theory.
When people don’t like what science tells them, they resort to conspiracy theories, mud-slinging and plausible pseudoscience
@Bart Verheggen #106
I am suspicious of any ontological system that claims to deliver unchallengeable truths. The extent to which scientists claim to have delivered such certainty is the extent to which they have perverted the real purpose of science, which is above all a rigorous but open-minded and dynamic system of inquiry.
From ““ “Hunting Down the Universe” by Michael Hawkins
For the last 12 years “climate science” has violated every part of that quote.
As for your quote – it’s not the sceptics who have generated the “ conspiracy theories, mud-slinging and plausible pseudoscience” , but rather the alarmists. It’s not the sceptics who have proposed the their opponents be jailed or committed or ignored, but rather the alarmists.
It IS the sceptics who keep on saying “show me the science”. And it’s the alarmists who keep on failing to do so. Keiths link (#99) fails entirely to present any real science. It is a compilation of simplistic, misleading factoids that fail entirely to prove the point and in some (many) cases are actually false.
I’m listening if you have anything useful to say. So… prove your case. But be aware that your points in #102 are generally without merit.
“”¦it’s the multiple lines of evidence that is key. Skeptics of AGW conveniently gloss over this, or ignore it.”
Here’s a great opportunity to embarrass those silly sceptics. Show them the lines of evidence that they have been glossing over or ignoring. They’ll feel like fools.
James Evans says: “”¦it’s the multiple lines of evidence that is key. Skeptics of AGW conveniently gloss over this, or ignore it.”
Here’s a great opportunity to embarrass those silly sceptics. Show them the lines of evidence that they have been glossing over or ignoring. They’ll feel like fools.”
=======================================
Er, no, they won’t. They’ll just gloss over this, or ignore it.
Sigh. Here and here and here and here and here.
[…] Comments keith kloor on What Is Science, Anyway?rustneversleeps on Where Science is FlawedJames Evans on Where Science is FlawedSashka on Fox’s Stage ManagementJim Owen on Where […]
@rustneversleeps #109
Your first reference starts with a quote from David Evans, but
takes that quote out of context. The “rest of the story” is as follows:
Although the models contain some well-established science, they also contain a myriad of implicit and explicit assumptions, guesses, and gross approximations””mistakes in any of which can invalidate the model outputs.
For a short lesson on models you might want to watch this:
http://www.crassh.cam.ac.uk/gallery/114
The speaker is Mike Hulme of the UEA and what he has to say is not consistent with your apparent view of models.
This is typical of the quality of “science” that alarmists generally present – partial quotes, with minimal or no explanation and, frankly minimal knowledge to back up their claims. I’m not impressed.
It was convenient that the first reference was about modeling since I have designed, built and operated models and I have a considerable knowledge of the pitfalls and limitations of such. Thank you for your consideration. If I have time I may get to your other references. I do sometimes enjoy the hunt for real science.
Lehrer falls into the same trap that he is warning about which is, poor experimental design and incorrect problem definition. This is ample evidence that there is climate change and this point is not in dispute. What is in dispute is the scientific work done which identifies CO2 as the main cause of this climate change and that further increses will lead to catastrophic global warming. It may surprise Mr. Lehrer that there is very little scientific evidence for this and alomost none of it has been replicatable in any degree.
rustneversleaps says:
“Sigh. Here and here and here and here and here.”
Oh you poor lamb. Sighing like that. I’ll take a look at all your links and I’ll report each and every ignored and glossed-over fact back to Sceptic HQ. But first I’m going to dig away all that snow from round my house. The snow that I was told I would never see again.
Orac tackles Lehrer’s New Yorker article.
Not that I believe that it will make any difference to anybodies opinion in this matter but here is my 10 cents:
I am biased – just as everyone else is in one form or another! However, I try to use science to mitigate these biased as much as is possible.
Would would love to believe that AGW is insignificant if for no other reason than I am extremely skeptical we are capable of doing anything about it. However, I am certainly willing to be presented with evidence I can believe to change this view.
I have spend some time researching the matter and as a result I am comfortable than doubling CO2 will creating a forcing excluding feedbacks of ~1.2C. So in that case I guess I am a believer in AGW?
Case closed? Well not quite. The question is not whether or not AGW is happening but whether it is happening to such an extent that it will insignificantly affect us and the planet? i.e. should we be spending huge amounts of money to solve? To answer this is any meaningful manner we need to know the following:
What is the effective of feedback? I understand that a huge amount of effort has been placed into trying to do this and many of these current models indicate positive feedback. I applaud these efforts but just have a couple of comments.
1) Right and in this matter is not decided by the number of models indicating a particular outcome – at best only one can ever be correct and the rest are simply wrong! 2) Sometimes is good to look out of the window to see what the weather is! The earth’s climate as been incredible stable of millions of years with relatively small and so changes. I understand that have been some exceptions to this (recent ice ages / PETM etc – which certainly warrant further investigation) but nevertheless we have a pretty stable world. Any control engineer / scientist would conclude that this system must be dominated by negative feedbacks. 3) The point of the article – everyone is biased – presumably this includes models and based on what they are trying to do it must be extremely difficult remove these biased from their work?
What is the general variability in the earths climate? Since temperature records have been kept we have detected a change of just under 1C. I assume some of this must have been due to AGW, however some must also be due to natural variability? This to me is an area of great uncertainly and the current data we have does not have the resolution to allow us to understand this. If the “Hockey” stick is not an example which meets the topic of this thread then I do not know what else is.
I fully admit that I have not studied all the articles involved but there are some facets of this “debate” I do find illuminating and from which I will draw my own conclusions (as will everyone else):
Science is a journey not a destination! I question any scientist claiming that the debate is over? If the case is so compelling I would welcome debate after debate after debate! Clearly one side of this discussion does not seem to what to engage with the other? One has to wonder why if the “science is settled”?
I am certainly willing to be presented with evidence I can believe to change this view.
I’ve said the same thing multiple times in multiple places.
If the case is so compelling I would welcome debate after debate after debate! Clearly one side of this discussion does not seem to what to engage with the other? One has to wonder why if the “science is settled”?
I’ve said the same thing multiple times in multiple places.
I wrote this on another blog yesterday – it fits –
That is, after all, what science IS ““ free and open debate with the objective of determining what the world is and how it operates.
I find too little of the “free and open debate” – and I often wonder what objective some people are pursuing. 🙂
[…] is why I retain strong faith in climate science as a process (keeping in mind that significant detours can persist in any area of science for quite awhile […]