What is your flavor of test format?

When it comes to testing - what is your favorite format? There are those who prefer the multiple guess format or the drag-and-drop questions. Then of course there is the "mark-the-right-area-with-the-red-X" type of test(in addition to multiple choice questions). These are all good ways of testing a person's knowledge of a product - but it becomes more a matter of memorization of minutiae then it is in knowing the how's-and-whys of a product. There is nothing directly wrong with the current testing methodology other than it is hard to filter the paper-certified individuals and those with real experience. The value of a certification is based on its' perceived worth in the IT community. If a certification is seen as not valuable because it does not adequately demonstrate the skills it is supposed to test, then it will have less value to an employer. Seems to me that the companies who are providing certification tests (Microsoft, Cisco etc) would be doing whatever they could to enhance the value of those certifications...more on this in another post.

I think the only true way of testing a person's knowledge is either by direct hands-on application or through a virtual environment. If you want to test a person's knowledge of a product in a way that shows they know the product vs. memorize facts of a product make them perform a series of tasks based on design criteria given to them and make them perform these tasks on a virtual machine. In this case, either they know the product and can perform the tasks given to them in the specified time frame or they can't. It will immediately separate those who just memorized information and those who really know the product and pass the hands-on test.

There are several drawbacks to this hands-on approach to certification, namely the flexibility of taking your test and the reliability of the network at the testing facility. Test development would change dramatically - the test developers would have to have virtualization skill or someone would have to convert their test into a virtual environment. How would the tests be graded - would it be on the successful completion of the tasks or would it be as much on "how" you did it as much as what you accomplished? These are all questions that would have to be addressed and discussed in the future.

Recent blog posts:

How do you prepare for a certification test?

Initial musings on Microsoft Certifications