Wikipedia-sponsored 'pilot study' lauds Wikipedia accuracy

Small sample size noted, but English Wikipedia said to score better than Encyclopaedia Britannica

Wikipedia logo

While Wikipedia's parent organization is taking great pains to emphasize the small sample size used in research it sponsored to spur others into more thoroughly assessing the online encyclopedia's quality and accuracy, there's no hiding its overall view of the results: We aced this test.

The Wikimedia Foundation last November enlisted the e-learning company Epic and researchers from Oxford University to conduct what would be the first organized look at Wikipedia's accuracy since a 2005 report by Nature that showed Wikipedia averaging four mistakes per article to three for Encyclopaedia Britannica. Companies and organizations sponsor self-serving research all the time, but rarely are they so open with the methodology and underlying data used, or enthusiastic about encouraging more thorough reviews.

From a Wikimedia Foundation blog post:

The small size of the sample does not allow us to generalize the results to Wikipedia as a whole. However, as a pilot primarily focused on methodology, the study offers new insights into the design of a protocol for expert assessment of encyclopedic contents. ...

The results suggest that Wikipedia articles in this sample scored higher altogether in each of the three languages, and fared particularly well in categories of accuracy and references. As the report notes, the English Wikipedia fared well in this sample against Encyclopaedia Britannica in terms of accuracy, references and overall judgment, with little differences between the two on style and overall quality score. Similar results were found when comparing Wikipedia articles in Spanish to Enciclonet. In Arabic, Mawsoah and Arab Encyclopaedia articles scored higher on style than Wikipedia, but no significant differences were found on accuracy, references, overall judgment and overall quality score. None of the encyclopedias considered in this study were rated highly by the academics in terms of suitability for citation in academic publications.

The full report can be read here (.pdf). An anonymized version of the data set is here.

The e-learning company Epic was equally thorough in describing the limitations of the research, but states in its own press release:

Nonetheless, Wikipedia articles in general emerge commendably in a number of respects, and it was possible to identify a pattern of qualities: Wikipedia articles were generally seen as being more up to date than other articles and were generally considered to be better referenced. Furthermore, they appeared to be at least as strong as other sources in terms of comprehensiveness, lack of bias and even readability.

 "We're very encouraged by the results for this small sample of Wikipedia articles," said Dario Taraborelli, Senior Research Analyst at the Wikimedia Foundation. "It affirms the quality of the editing community's collaborative work and it provides valuable methodological insight for future studies."

Encyclopaedia Britannica did not agree with the findings of that 2005 Nature report. I have contacted its press office to see if it will have anything to say about this one.

(Update: A Britannica spokesman tells me they received no advance notice or copy of the research. I assume they'll have something to say once someone there has had a chance to look at it.)

Welcome regulars and passersby. Here are a few more recent buzzblog items. And, if you'd like to receive Buzzblog via e-mail newsletter, here's where to sign up. You can follow me on Twitter here and on Google+ here.

Editors' Picks
Join the discussion
Be the first to comment on this article. Our Commenting Policies