For the first time ever, companies that develop and administer IT certification exams are working together to combat a problem that has largely been swept under the rug for years: certification fraud.
A group of IT hardware and software vendors, independent certifying agencies, test centers and others have formed the IT Certification Council (ITCC). The goal is to share knowledge and resources to combat and prevent fraud, which is threatening to undermine the value of IT certification. (See Musthaler's opinion on how cert cheating can ruin your career.)
ITCC chairman Bill Horzempa, who is also director of Global Certification and Partner Education Development for HP, says, "Most of the members of this council have talked privately with one another about the cheating problem. We realized that this isn't just an HP problem, or a Cisco or Microsoft problem. Certification cheating affects the vendors, yes, but it also hurts individual IT professionals and the companies that employ or contract them. In effect, cheating creates a loss of confidence in the ability of the IT profession to solve business problems."
|What is certification fraud?|
“Certification fraud is any act, malicious or not, that is intended to help an exam candidate pass a certification exam using methods that violate vendor security policies.” This includes acts perpetrated by certification candidates and corrupt test center proctors, as well as by the individuals and organizations that post and sell ill-gotten test materials on the Web.
Julieann Scalisi, Managing Director of Citrix Education
Chuck Cooper, ITCC vice chairman and program director, IBM Certification Programs Skills Enablement, Systems and Technology Group, calls certification fraud "an annoying pain that always seems to be there. It's a cloud hanging over us. It doesn't go away on its own."
Indeed, fraud in the IT certification industry is nothing new; the problem has been around for years. However, new techniques for analyzing test scores are making it easier to evaluate the scope of the problem. For example, test security company Caveon estimates that 15% to 25% of IT certification exams show some aberration, which can be an indication of cheating.
Ignoring the problem has only allowed it to get worse. All one has to do is Google the search term "MCSE study aids" and thousands of sites pop up where a student can purchase so-called test preparation materials – most of which are not authorized or recommended by Microsoft, the owner of the MCSE certification.
Though the documents are marketed as "study materials," the information often consists of stolen test questions and answers. Of course, Microsoft isn't the only company whose materials have been compromised. Content for virtually any IT certification exam can be found online.
The impact of certification fraud
Certification cheating has ramifications for everyone, including the individuals who pursue certification; the employers who hire them; the companies that contract for IT solutions and services; the IT vendors who manufacture and sell IT products and solutions; the certifying companies and agencies; and more broadly, the general public.
|What happens if you get caught cheating?|
The individual who cheats is taking a risk with his career. If students are found to be cheating, they can face a range of consequences, such as negation of their test results; loss or denial of certifications; banishment from a certification program; or notification to his employer. Each certifying agency sets its own security policy which should be understood before a candidate undergoes the certification process.
Employers also suffer when individuals cheat on certification and are not truly qualified for a job. "If employers aren't getting quality work out of their employees, they are being defrauded," according to Taylor Ripley, chief security officer, CertGuard. "Employers need to know they are getting what they ask for."
Ripley says the companies that are most likely to suffer damage from certification fraud are the smaller size companies that don't have a Human Resources department to help weed out people who can't do a job. "These companies are forced to rely on certifications to judge a person's qualifications. A small company could lose money or business if an unqualified person screws up," Ripley says.
The VAR factor
But sometimes it is employers who encourage employees to get certified using any means necessary. For example, a systems integrator or value-added reseller (VAR) might want to get authorized to sell a particular vendor's product. Authorization might require that the company have one or more certified professionals on staff.
"If a VAR helps his employees cheat to get a certification in order to get or stay authorized, the company's customers are affected, as well as the vendor that the VAR represents," Ripley says. "Say someone cheats to reach the Microsoft Gold Certified Partner level. If the VAR implements a poorly designed solution, the customer has wasted his money and he thinks Microsoft has bad products. Everyone loses when this happens."
Rick Gregory, managing director of the training community of TrainingIndustry.com, has heard of instances in which outsourcing contracts are being canceled and the work is being brought back in-house because the people assigned to the contract simply weren't qualified. "The contract specified a requirement for specific kinds of certified professionals, so the people went out and purchased a credential," Gregory says. In the end, the work was below standards set in the outsourcing agreement.
Vendors such as Microsoft and Cisco and third-party agencies like the Computing Technology Industry Association (CompTIA) and the Storage Industry Networking Association (SNIA) that sponsor certification programs lose both money and intellectual property when even one exam is compromised. It can cost hundreds of thousands of dollars and take numerous subject matter experts three to six months to develop a certification test.
"We hear from candidates that some of our tests are readily available," IBM's Cooper says. "It's a compromise of our [intellectual property]. Our internal sponsors wonder about the validity of the tests. They typically don't need to rewrite the tests, but they need forensics to understand the impact to the test scores. Nevertheless, the perception is that damage has been done."
Fraud and the countermeasures
In order to develop measures to combat fraud, the certifying agencies need to understand how cheaters operate. Here are some of the cheating techniques that have been identified and what authorities are doing to thwart the fraud.
One of the oldest tricks in the book is to get someone else to take the test in place of the real candidate. Called a proxy test taker, a person goes to a test center and takes an exam registered as someone else. A few "entrepreneurs" have even turned this technique into a business.
"Recently we found that our certifications, along with other IT certs, were being sold on the Internet via a proxy test taking service," says Julieann Scalisi of Citrix. "Caveon, as part of our new Web patrol service, took the action to have them removed from Google. Unfortunately, the site still exists and they appear to be selling Citrix certifications from $700 to $4,800."
Cisco and test delivery company Pearson VUE are in the forefront of implementing stringent candidate authentication techniques to discourage proxy test taking. Soon, each Cisco exam candidate will be required to have a digital photo taken at the test center, and must provide a digital signature in order to take the exam. The photo and signature will be attached to the test results.
Over time, Cisco and Pearson VUE will be able to spot individuals whose photos appear under different names and signatures. Other vendors and test delivery companies are exploring the use of biometrics such as fingerprints to determine if one person is taking tests under numerous names.
Erik Ullanderson, manager of Global Certifications for Learning at Cisco, is happy to share his antifraud techniques with his colleagues on the IT Certification Council. "Our efforts in curtailing fraud are not a Cisco-only value-add," Ullanderson says. "We think other companies should be jumping on the investments that Cisco and Pearson VUE have made." Indeed, the ITCC is looking at how it can utilize this and similar programs worldwide in light of privacy concerns in various countries.
Another common cheating technique is to have the test items and answers in advance. Such information is often posted to certification forums, blogs or brain dump sites, giving a candidate the opportunity to memorize rather than actually learn the subject matter. "We know that exam content can be found on different Web sites for a fee," Scalisi says. "Content and answers also can be found within blogs and discussion forums that are usually intended to help others answer difficult exam items, sometimes providing hints but often times providing actual answers."
More blatant are the Web sites that sell hundreds of actual exams, marketing them as study aids. "Certification candidates need to know that certifying agencies never provide their exams or other preparation materials to these brain dump sites," HP's Horzempa says. "Most of what is posted has been obtained through illegal means." In fact, brain dumps are often a violation of the laws protecting copyrighted intellectual property.
Targeting the consumer
This, then, begs the question: why don't authorities shut down the brain dump sites? Because it's not as easy as it seems.
"In the late 1990s, the Digital Millennium Copyright Act (DMCA) gave software companies and testing centers the ability to go after unauthorized providers of test content," says David Meissner, vice president of Solution Services at Prometric. "But having this legal tool doesn't make it easy to go after the offenders. Often they are located in countries that don't recognize U.S. laws, making prosecution difficult to impossible."
It takes very deep pockets to pursue the purveyors of brain dumps. Civil or legal action can drag out for years with little success to show for the effort. Many certifying agencies will pursue a cease-and-desist order rather than a lawsuit if their intellectual property is compromised.
A different strategy for combating certification cheating is to go after the consumer of the illicit materials. "Brain dump sites are like drug dealers," says Lee Futch, product management lead for Symantec Education Services. "As long as there is a customer, there will be a dealer. We need to cut off the customer base to kill the illegal dealers of stolen [intellectual property]."
One of the missions of the ITCC is to spread the word to candidates that the certifying agencies are indeed going after the consumers of the stolen test materials whether the consumption was intentional or inadvertent.
The good news is that it's getting easier to spot cheaters. Using new data forensics techniques that didn't exist just a year or two ago, certifying agencies now collect metrics that can indicate the possibility that someone has used illegal tactics to pass the exam.
The metrics reveal statistics such as how long it took the student to answer each test item, which answers were changed during the test, and how much time the student needed to complete the test. These metrics are compared with a historical baseline value, and too much variation raises a red flag. Before the student even walks out the door of the test center, the test results can be called into question, triggering further investigation.
Even "inadvertent cheaters" can be caught this way. People who use information from the brain dump sites are essentially able to memorize or at least practice actual test questions and answers, whether they do it knowingly or not. This advantage can be readily identified in the test metrics, and the candidate can be singled out for further investigation and possible consequences.
"Citrix uses data forensics to identify specific instances of cheating," Scalisi says. "We now conduct a monthly review to identify anomalous scores and results. Once confirmed as cheating, candidates are subject to remedies up to and including certification revocation and ban from testing for up to one year."
ITCC members don't share data forensics about specific exams or individuals, but they do share information about testing centers if corruption is suspected. "Forensics let us look across tests and centers around the world," IBM's Cooper says. "When a test center appears to be compromised, we gather statistically valid proof to act upon. This data is based on tens of thousands of tests that are administered each year."