A Teachable Moment: Open Source Platforms for Online Testing

A new study says there might be room for open source testing software in statewide assessments.

As more states begin to consider moving assessments online, they're also considering whether to develop the testing platforms on an open source base.

Some inroads already have been made - Utah has been using a statewide open-source platform to support its formative assessments since the early 2000s. Formative assessments are, more or less practice exams. They aren't used to determine funding or grades, but rather to help teachers determine how well the students are absorbing the material and what they need to focus on, as well as giving students experience in taking such tests.

The Educational Testing Service contracted with Grunwald Associates LLC to conduct a study of educators around the nation, "An Open Source Platform for Internet-based Assessment: A Report on Education Leaders’ Perceptions of Online Testing in an Open Source Environment," which was released today.

Utah ended up using open source more by accident than design. A group of technology consultants working with rural districts in the state were asked by teachers in those districts to help create some online practice tests. "These particular technology consultants had already been working with an online open source learning management system, which they adapted to create an application wherein teachers could enter their own test items into an online interface that would then generate practice tests for their students," according to the report.

The state applied for and got a federal grant to broaden the system and scale it for the entire state, with some additional quality controls to ensure all questions aligned to the curriculum and had been reviewed. The teachers stayed involved, though, making it open source on both the software and the input levels.

The study came to five conclusions about what would have to happen in order for more states to get involved in online testing software built with open source code:

• People need to be informed: Basically, most education leaders don't know enough about open source to make an informed decision on what platform to use.
• Level of interest may be related to the current level of investment: This one's a matter of simple economics. States that already are doing online testing are less likely to switch platforms if they've invested a lot in developing it. The one exception is if their testing publisher were to make the switch.
• Money is important, but it's not everything: Quite simply put: Quality is more important. Security, support, expertise - when it comes to statewide online testing, these issues take an extra level of importance.
• It's hard to be first: Of course, Utah's already kind of there, so that takes a little of the pressure off. But - just like in any other industry, educators aren't so comfortable being trailblazers.
• Everyone wants structure and organization in their coalition: They "need to see examples of open source communities that were able to successfully work together to create complex, high-stakes products that met the needs of all the members of the community."
• Complext test items might be a way in: Using the open source platforms to go beyond simple multiple-choice exams would give the OSS community a leg up on the competition.
• Leadership is key: If someone is going to grab hold of this and bring the educational community of the U.S. around to open source, he, she or they will have to clearly show an understanding of different states' needs and how to manage the politics.

Of course, many of those points above could apply to nearly any industry, community or business. States are highly competitive with one another when it comes to who has the best test scores, and don't always play nice with one another.

But educators also sometimes can be more collaborative than others and often are more willing than others to share best practices. If they can get past the politics and concentrate on how best to use online test building and testing to, well, educate, they could, perhaps, use the very platform itself as a way to educate their students on how to work together.

Sounds like a teachable moment to me.

• Software
• assessment
• education
• open source
• testing