Fad or Revolution? A Look at Science 2.0

When Thomas Kuhn wrote his theory of scientific revolutions it was a revelation both to the debates surrounding the philosophy of science and to researchers and scientists as well. Kuhn argued that instead of science through a process of hypothesizing—testing—reporting—interpreting it proceeded instead through shifts in paradigms. Paradigm shifts would arise when debate around anomalies and inconsistencies in the results of an established research agenda (perhaps a set of methods centered around answering a certain kind of question i.e. using roll-call data to analyze the behavior of the US Congress) reached a certain pivotal point. That is, when the anomalies and consistencies attracted a critical mass of attention, analysis, and research of their own leading to a paradigm shift with the new paradigm organized around the response to the anomalies in the previous paradigm.

This crude summary of Kuhn is provided here to serve as the backdrop for something I have become increasingly interested in—the concept of Science 2.0 . Like any of the other “2.0” terms out there (Government 2.0 , Web 2.0 , etc), Science 2.0 is open to many interpretations. Here, I think of Science 2.0 in perhaps its broadest sense—harnessing the power of new communications technologies to promote transparency, openness, and collaboration in scientific research.

The question that I immediately asked myself upon reading Kuhn’s theory of paradigm shifts was: Are we on the verge of a paradigm shift from Science 1.0 to Science 2.0, or is Science 2.0 merely a fad? Of course the answer is yet to be determined, but looking at the potential of Science 2.0, impediments to its spread throughout the scientific community, and its pitfalls can give us a good understanding of the concept. To do this I will draw on examples from my home discipline of political science, but most of the issues I discuss could be generalized to any scientific endeavor.

First, let’s talk about what is wrong with Science 1.0. As a graduate student, and thus a fledgling researcher, the most frustrating impediment to the advancement of knowledge is the lack of transparency in methods and data. The scientific method is reliant on the idea that to accept a finding a researcher’s work has to be able to be replicated by an independent, neutral, and unaffiliated researcher. In practice though this is not always feasible. Whether it is because the researcher is merely paying lip-service to the notion of verification (by vaguely describing the methodology and data used) or due to a lack of resources (because the research was conducted as part of a massive study that other researchers could not find time/money to fund simply to fact-check another researcher) much research goes unverified. While the possibility of verification may deter unscrupulous researchers from outright manipulation of data and results (though not always…) it does not mean that inconsistencies, anomalies, and idiosyncrasies to a particular run of an experiment or data analysis are ruled out. Indeed, within published books in political science there are statistical inconsistencies, typos, and methodological errors that occur. When they are caught, that is great, but how can we be confident that a system that is not transparent is catching them all? ¹

A recent article in Scientific American points out the benefits of collaboration in a vague way, it glosses over the issue of verification and replication. Within political science it is normal for authors to offer replication data sets upon request to researchers interested in replicating/augmenting their work, but how much replication data is provided, in what format, in what timeframe, and the restrictions on the use of that data are all at the discretion of the author. ²

Why? Authors are worried about “scooping”—another researcher mining their data and finding some important/interesting result that leads to a publication. And it is hard not to blame them—as a commenter on a related Scientific American article points out this is a much greater risk than someone reaching their same conclusion before them. Within the social sciences a single dataset can often yield several articles/book chapters worth of interesting results and releasing such a set early would put added pressure on a researcher to mine that data and generate results faster than others who have access to the same data. In a world where publications can make or break tenure it is understandable that researchers are protective of datasets that could provide them a series of publications over the course of a few years.

So the problem is two-fold. 1) Researchers are increasingly conducting research that is difficult to verify from the merits of the publications that research appears in alone, and 2) Researchers are incentivized away from being more transparent by the reward system baked into the structure of academia. This system has created a culture of secrecy, obtuseness, and impenetrability within the sciences. The last vestige of verification—peer reviewing of published articles—is becoming increasingly unable to cope with the sophistication of research conducted.

Yet, without verification and fact-checking by other researchers we cannot claim to be producing knowledge, because what is produced is not known to be true with any level of certainty. A recent Wired article provides an example from the field of paleontology. After a spirited academic debate fostered by the publication of all results and data on an analysis of a fossil the original (and disputed/controversial findings) of one researcher were accepted with much more confidence and researchers were able to focus their efforts around further verifying/refuting this finding.

Science 2.0 is not ready to take off for another reason—to really work it requires a critical mass of users. The percentage of scientists in any given field participating online or using a given collaboration tool is quite small. A big factor in this is that there is no “credit” given to scientists who blog or post to wikis online (as SciAm points out). Moreover, publishing something online that is incorrect, inflammatory, or unwisely phrased could seriously jeopardize the potential for a faculty member to receive tenure. Sharing thoughts, ideas, and musings online not only takes time away from academic pursuits more closely tied to direct rewards (publishing articles), but it also exposes a faculty member to extra risk in a competitive job market.

But, what is the scientific community losing by creating such incentives? Think of the life of a typical journal article. After a long and torturous gestation period on the hard drive of one or a few researchers, a brutal submission process, and the recommendation from 2-3 reviewers for publication an article appears in a journal (several months after completion of the original submission and possibly several years after completion of the data collection and analysis). Researchers in the field pick up a copy, read it, discuss it with one another around water coolers and over e-mail with close colleagues. If the article is particularly controversial and speaks to a contested debate within a discipline there may be a response article that appears in the next issue with a rebuttal from the author. Perhaps the article will be cited by other scholars in the field as a jumping off point or a false start that their own research makes use of, and eventually the article is forgotten.

Think of the alternative. An article is published in a journal. It appears online where all journal subscribers are invited to post public comments with questions, comments, and critiques of the author’s argument, data, and methods. The author responds to these comments and a spirited academic debate ensues. Using data made public by the author available on the same page researchers report inconsistencies in the findings or question the details of the work done by the author. Out of this discussion the author identifies further research that could bolster his/her overall theoretical claim. Other researchers identify pitfalls of a particular methodology and are at once more reassured in the validity (or invalidity) of the author’s claim. The discussion happens instantaneously and trivial academic disputes are resolved quickly so that the research that is rewarded is truly innovative and groundbreaking, and based on valid propositions developed by researchers that came before.

This is the power of Science 2.0. Science speeds up while at the same time becoming more accurate and more rigorous. There will be tradeoffs of course, and the problem of scooping must be resolved, but researchers need to be pushed to embrace the openness and transparency available through new technology because it more closely adheres to the scientific ethic of pursuing and producing “knowledge” (versus conjecture) that they claim to embrace.