Open source returns integrity to science

No readers like this yet.
A bubble in liquid

Opensource.com

Imagine it is 1912, but that the Titanic is fitted with an underwater radar system. Imagine that it senses an iceberg so large that even the captain can understand that by the law of conservation of momentum, the ship will be stopped in its path. Should the captain use the radar information to inform the decision to alter course, or should the captain ignore it because radar is merely an invention of science therefore prone to exaggeration and false findings?

The New Yorker Magazine has just published an immensely popular article titled "The Truth Wears Off -- Is there something wrong with the scientific method?" The article reports several examples of scientific findings that appeared to be significant when first published, but when tested over time, demonstrate weaker and weaker results. Zyprexa is a second-generation anti-depressant that showed great promise in clinical trials in the nineteen-nineties. By 2001, Zyprexa earned more revenue than Prozac, and it remains Eli Lilly's top-selling drug.

According to the article, recent studies of these second-generation anti-depressants show that the therapeutic power of the drugs appears to be steadily waning, down to less than half of that documented in the first trials. It reports that many researchers began to argue that the expensive pharmaceuticals weren’t any better than first-generation antipsychotics, which have been in use since the fifties. And it quotes John Davis, a professor of psychiatry at the University of Illinois at Chicago, as saying “In fact, sometimes they now look even worse." How could such drugs be approved if the FDA is using the scientific method, which requires independent reproducibility of results?

Quoting the article:

But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology. In the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants: Davis has a forthcoming analysis demonstrating that the efficacy of antidepressants has gone down as much as threefold in recent decades.

In 2005, a paper was published titled "Why Most Published Research Findings Are False," and it has become that author's most-cited work. If we paradoxically accept its findings as true, what reasonable interpretation should we give to President Obama's inaugural promise to "restore science to its rightful place"? This is a serious question  If we read the science on global climate change, it reads like a radar screen flashing RED ALERT about the impending iceberg of environmental collapse. Should we heed the warnings that more than 95% of all climate science papers report, or should we maintain course, confident that all these predictions are nothing more than statistical aberrations and gamesmanship?

Over the New Year's holiday I had a chance to watch the movie "Fair Game" [official website: www.fairgame-movie.com], which is based on Valerie Plame Wilson's book "Fair Game: My Life as a Spy, My Betrayal by the White House". (It also incorporates material from Ambassador Joe Wilson's book "The Politics of Truth: Inside the Lies that Led to War and Betrayed My Wife's CIA Identity".) In that movie, the subject of truth is examined in many contexts.   There are the 16 words that George W. Bush uttered during his January 28, 2003 State of the Union address ("The British government has learned that Saddam Hussein recently sought significant quantities of uranium from Africa."), words accepted as true enough to authorize the invasion of Iraq. There is the op-ed piece that Ambassador Wilson wrote on July 6, 2003 ("What I didn't find in Africa").  There is a dramatization of the discussion between CIA and the Office of the Vice President about intelligence and whether the seriousness of a potential scenario (Iraq acquires fissile nuclear material) should be allowed to influence the assessment that such facts are in evidence. Finally, the movie shows how the selective use of source information, without proper process, led to a clearly erroneous assessment of facts regarding Iraq's nuclear capabilities, not to mention the possibility that erroneous decisions were made in light of those facts.

It is estimated that the NIH spends more than $30B/year on medical research, and that the CIA spends more than $40B/year on intelligence activities. If "most published research findings are false," would we all be better off in a pre-scientific world? How are we to make policy and investment decisions, be they question of which armies to raise against rogue nations or which drugs to take against rogue cells?

The New Yorker reports that in a forthcoming paper, Jonathan Schooler recommends the establishment of an open source database, in which researchers are required to outline their planned investigations and document all their results. This is an interesting prospect, especially because of the studies I've read about open source software.

The IT industry spends $1.5T/year knowing full well that $500B/year is being wasted on software and systems that will never make it to production, or if they do, bad software quality will "challenge" them with schedule slips, missing features, and bugs significant enough to interfere with operational capabilities. The #1 reason for choosing open source software in the enterprise is "quality as compared with proprietary software". In a series of studies published by Coverity, open source software has achieved on average (across more than 250 projects, more than 55 million source lines of code (SLOC)) 100x lower defect density than proprietary software. Is any of that true? Or is it just a bunch of scientific nonsense?

I believe that the answers to these questions are among the most important we face today. What is science? What can it know? What can it teach us? How should we make decisions based on that information? I'm excited to discover that others believe that open source, which has been inspired by the scientific method, may yet be called upon to rescue science from those who merely try to confirm pre-conceive hypothesis. Can open source prove itself to be valuable? That would be quite a feat. But honestly, I see no better course.

User profile image.
Michael Tiemann is a true open source software pioneer. He made his first major open source contribution more than three decades ago by writing the GNU C++ compiler, the first native-code C++ compiler and debugger. His early work led to the creation of leading open source technologies and the first open source business model.

5 Comments

From the article: "Should we heed the warnings that more than 95% of all climate science papers report, or should we maintain course, confident that all these predictions are nothing more than statistical aberrations and gamesmanship?"

The climate change movement is a political movement, not a scientific study. Climate change advocates take any perceived change in the biosphere and create a chain of logic, no matter how contrived, to "prove" that the change is due to global warming. In fact most of the changes that are reported as proof of global warming have nothing to do with global warming and neither prove nor disprove global warming. Most of the political editorials trying to pass themselves off as scientific papers are junk science.

Another problem with the climate change political movement is they start out with the rock solid assumption that global warming is an evil thing. No consideration is given to the possibility that global warming might lead to an increase in the amount of life on this planet. Indeed there have been periods of time in the Earth's history when the Earth was significantly warmer and the total biomass was significantly larger than today. Having lived in Canada for seventeen years I have the opposite bias. I think that global warming is a good thing and it should be encouraged.

-----------------------------
Steve Stites

My article treated the subject about how to make science better, i.e., by recognizing where it has failed, and strengthen it with new methods based on open source tools.

Your response lacks any specific evidence for your viewpoint that climate change science is bunk, and goes on to wishfully believe that human-caused climate change is a good thing, without any sort of trade-off analysis (or reference thereto). If your last paragraph is correct--that humans do have the ability to cause global climate change, then you do seriously undercut your own argument that there is no global climate change and that there is no knowable linkage between human activity and such change. That does not seem very scientific to me.

"If your last paragraph is correct--that humans do have the ability to cause global climate change, then you do seriously undercut your own argument that there is no global climate change and that there is no knowable linkage between human activity and such change. That does not seem very scientific to me."

I do not argue that there is no global climate change. I argue that this vast amount of scientific research you cite is junk science created to advance a political cause. My answer is a political answer to a political topic.

As to citing cases of junk science in the past I have rebutted:

An article in Slashdot that stated that the melting arctic ice pack would cause global flooding.

An article in Scientific American which stated that the author deduced from the fur harvest figures that a decline in the Wolverine Canadian snow pack coverage has caused a decline in the number of Wolverines in Canada.

That melting arctic ice has the potential to increase the size of the Labrador current to where the Labrador current would block the Gulf Stream. Published in several online news blogs.

I will take it as a given that you have read a large number of articles presenting scientific evidence of global warming if you will accept the fact that I have read the same articles and dismiss most of them as junk science. And yes there is actually some scientific evidence for global warming which is lost in the midst of all the nonsense printed by the climate change political movement.

The scientists who have done the decent work on global warming have taken the non-scientific approach of placing a value judgment on their findings. This launched the climate change political movement with an unquestioning judgment that global warming is evil. I make the political judgment that global warming is a good thing. Neither my judgment nor the climate change movement's judgment is scientific.

------------------------------
Steve Stites

You clearly state that when it comes to political discussions, there is no place for science. You give an example of the science of global climate change, which may have had some validity as a series of measurements that make no predictions, but which (in your view) lose all scientific validity the moment they are used to make predictions or, worse, potentially inform policy. But your argument generalizes immediately to virtually all science.

I have witnessed 40+ years of scientific progress--successful predications based on theories which, over time, are essentially accepted as truth: how electrons flow in semiconductors, how images can be derived from scattered radiation of sound and light waves, how inductive reasoning can lead to the development (and interpretation) of programming languages, etc. As the sources I cite in my article show, science can easily be derailed by people chasing significance rather than publishing theories and openly testing their hypotheses. But I disagree fundamentally that science can never be trusted to inform policy. I believe that unscientific and anti-scientific policies underlie some of the worst problems we face today.

As Albert Einstein once said "We cannot solve our problems with the same thinking we used when we created them." Sound science represents one kind of new thinking that needs to be tried.

The claims made by those on both sides of the global warming debate are propaganda - deceptive statements of fact designed to provide convincing evidence to support their side of the debate. specific example:

fact (even quoted by gore): Greenland ice is melting at a net rate of 330 cubic kilometers per year.

misleading because at that rate, or even faster, it will take at least 10,000 years to melt all the ice on Greenland. why doesn't anyone say this? because the melting of Greenland ice would no longer be a problem.

It is easy to show that every claim made is propaganda of this nature. give me a claim and I'll try.

The nature of science should be "full disclosure," I.e., open, but it is not.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.