A Band-Aid for Science
Wednesday, March 19, 2014
Here's a sign of the
times: The man who once published a paper titled, "Why Most Published Research Findings Are False", and
others have formed a new institute to police bad science. In addition to bogus
results, the institute (METRICS, for "Meta-Research Innovation Centre at
Stanford") will also consider the problem of wasted effort:
... A recent series of articles in the Lancet noted that, in 2010, about $200 billion (an astonishing 85% of the world's spending on medical research) was squandered on studies that were flawed in their design, redundant, never published or poorly reported. METRICS will support efforts to tackle this extraordinary inefficiency, and will itself update research about the extent to which randomised-controlled trials acknowledge the existence of previous investigations of the same subject. If the situation has not improved, METRICS and its collaborators will try to design new publishing practices that discourage bad behaviour among scientists. [bold added]While this effort is laudable, I think it will fail, because I think many of these problems are ultimately due to government funding of science. Indeed, it reminds me a little of recent efforts to address the problem of the government training too many new scientists in certain fields -- such as by adding a whole new training program to their terminal degrees.
Set aside, for the sake of argument, the whole question of whether the government should be taking money forcibly from some citizens for any reason. What we are seeing in science, as with many other areas of the economy, is a vast amount of money being poured in to an industry by a political class whose primary motivation isn't the discovery of truth (rather than flowing there due to market forces) and creating a whole slew of artificial, perverse incentives. When these perverse incentives guide individuals, it should come as no surprise that it is common to see examples of money being wasted, be it in the form of sloppy studies, redundant work, or over-training. It may be possible to mask or slightly alleviate the symptoms for a time, but the disease goes merrily on.
-- CAV
4 comments:
Politicians will fund a flawed study that will give them the answer they want so that they can cite it later when arguing for some policy or other. They and the study's "scientists" know full well that the results are false, or flawed in some way, but they're counting on voters' going blindly along with the appeal to authority. I'm not optimistic either, but I hope METRICS has at least some success.
I hadn't considered the prospect of METRICS (or something like it) looking at AGW, but the idea of it is appealing. However, it appears that the institute will focus, at least at first, on medical research.
Whether something like it in other fields, such as meteorology, would be permitted to exist for long, unless funded independently of government, is an interesting question.
On the face of it, I like the idea of such an institute. On nutrition advice, for instance, the popular press will report a study that says one thing, and then a study that says the opposite.
I think it is because the press has failed in its QA mission. The typical reporter has been taught that objectivity comes in reporting all sides -- not from the reporter's own mental evaluation. So, when there is only one side pushing a story -- e.g. a researcher -- the counter-argument gets scant attention.
It would be nice to have someone doing private and independent QA. Perhaps the press would then turn to the QA body for a second opinion.
Of all things, the way CNN responded to the recent disappearance of the Malaysian airline plane has alerted me to the poor intellectual state of people in the media. The other day, Don Lemon said that some people are asking if we should now consider super-natural explanations! He asked another interviewee if it would be preposterous to think a black-hole could be responsible. This is from one of CNN's lead anchors: that tells me that media reporting of scientific topics no longer has a slightly more objective basis than its reporting on ethics and politics.
RT,
You make an excellent point, and the analogy to things like Underwriters' Laboratories and the Consumers' Union is both apt and prescriptive.
In a scenario in which science funding were less dominated by the government, this is what we'd want, and probably have. But we aren't in that scenario, so I see a couple of problems right off the bat. First, improper government has probably made science "bigger" in terms of how many workers there are, but not in terms of the quality of the work (e.g., the figure from the Economist article). Just how much can this institute keep an eye on? (But something is better than nothing, and could be exemplary.) Second, if this Institute has government funding, I can envision its mission being compromised. (This doesn't mean it WILL happen.)
That said, I am glad you spoke up. Much needs changing in the world, and the change has to start somewhere. Perhaps this can be one of those places.
Gus
Post a Comment