While Wikipedia's parent organization is taking great pains to emphasize the small sample size used in research it sponsored to spur others into more thoroughly assessing the online encyclopedia's quality and accuracy, there's no hiding its overall view of the results: We aced this test.
The Wikimedia Foundation last November enlisted the e-learning company Epic and researchers from Oxford University to conduct what would be the first organized look at Wikipedia's accuracy since a 2005 report by Nature that showed Wikipedia averaging four mistakes per article to three for Encyclopaedia Britannica. Companies and organizations sponsor self-serving research all the time, but rarely are they so open with the methodology and underlying data used, or enthusiastic about encouraging more thorough reviews.
From a Wikimedia Foundation blog post:
The small size of the sample does not allow us to generalize the results to Wikipedia as a whole. However, as a pilot primarily focused on methodology, the study offers new insights into the design of a protocol for expert assessment of encyclopedic contents. ...
The results suggest that Wikipedia articles in this sample scored higher altogether in each of the three languages, and fared particularly well in categories of accuracy and references. As the report notes, the English Wikipedia fared well in this sample against Encyclopaedia Britannica in terms of accuracy, references and overall judgment, with little differences between the two on style and overall quality score. Similar results were found when comparing Wikipedia articles in Spanish to Enciclonet. In Arabic, Mawsoah and Arab Encyclopaedia articles scored higher on style than Wikipedia, but no significant differences were found on accuracy, references, overall judgment and overall quality score. None of the encyclopedias considered in this study were rated highly by the academics in terms of suitability for citation in academic publications.
The e-learning company Epic was equally thorough in describing the limitations of the research, but states in its own press release:
Nonetheless, Wikipedia articles in general emerge commendably in a number of respects, and it was possible to identify a pattern of qualities: Wikipedia articles were generally seen as being more up to date than other articles and were generally considered to be better referenced. Furthermore, they appeared to be at least as strong as other sources in terms of comprehensiveness, lack of bias and even readability.
"We're very encouraged by the results for this small sample of Wikipedia articles," said Dario Taraborelli, Senior Research Analyst at the Wikimedia Foundation. "It affirms the quality of the editing community's collaborative work and it provides valuable methodological insight for future studies."
Encyclopaedia Britannica did not agree with the findings of that 2005 Nature report. I have contacted its press office to see if it will have anything to say about this one.
(Update: A Britannica spokesman tells me they received no advance notice or copy of the research. I assume they'll have something to say once someone there has had a chance to look at it.)
Welcome regulars and passersby. Here are a few more recent buzzblog items. And, if you'd like to receive Buzzblog via e-mail newsletter, here's where to sign up. You can follow me on Twitter here and on Google+ here.
- Microsoft apologizes for phrase "big boobs" in Linux code
- Techie talks about life of lies ... and his recurring need to leave
- Snopes.com debunks old C++ hoax, but ...
- Microsoft needs to buy a vowel.
- Watch Steve Jobs play FDR in long-lost Apple take-off on "1984"
- 2012's 25 Geekiest 25th Anniversaries.
- Googler on why he left Google ... for the third time.
- HP's policy on politics: "Do as we say, not as Meg does."
- Cool eclipse video needed no "Star Wars" special effects
- History's first prank call almost as old as the telephone.
- "Hilarious Unix admin tools"
- Steve Jobs and his gadgets ... in LEGO.