On Certifications, Knowledge, and Competence

There is an ongoing and more recently growing controversy surrounding the ways that testers and other software professionals demonstrate their competence. In particular, the controversy centers on the testing services provided by the world’s largest provider of certification services, the ISTQB, and its US arm, the ASTQB.

The primary criticisms of the ISTQB seem to be that the use of a multiple choice test for certification trivializes testing knowledge and judgment. In addition, some see the tutorial offerings of supporters to be an unnecessary and unwanted commercial advantage in training for this specific certification. So there are currently online petitions that demand the ISTQB release its own data that assesses the validity of the certification.

While both of those criticisms have some truth, Cem Kaner notes in a comprehensive post that the ISTQB seems intent on measuring and improving its certifications, and supports its right to examine their data and institute those improvements in private. As long as it is not making demonstrably false statements regarding the knowledge gained as a result of certification, the organization has the legal and ethical right to improve its exams. Kaner does not claim to have reviewed all of the organization’s marketing materials, but does assert that he sees no apparently false claims.

Kaner, one of the true deans of software testing, is right. I have referred to him as our “adult supervision” on Twitter. But the argument goes beyond his level-headed analysis of what the ISTQB offers and does not offer to testers.

We are fortunate to work in a field where there are a lot of ways of adding value. Some write code to build software, and write unit tests to verify how they think discrete parts of the code should work. Others work closely with developers to understand the underlying structure and develop tests that reflect the requirements of the application and its underlying requirements. Still more are domain experts, fighting for the user community and possessing a keen understanding of what is needed beyond the stated requirements to deliver a quality product.

Measuring all of those skill sets in a single exam, whether multiple choice, single correct answer, or even in essay format, seems incongruous and indeed impossible. Yet a broad range of people and skills can and do help in determining whether an application meets the needs of the business and has the necessary quality to deploy.

Part of the larger problem is that some employers expect ISTQB certification as a prerequisite for a testing job, which distorts its value in the marketplace. Another part is that ISTQB marketing, while not demonstrably false, might be interpreted as misleading.

Kaner says that while the ISTQB certification lacks many essentials, we have not yet been able to devise anything better. He’s not happy with ISTQB certification, but for technical rather than business reasons. And he’s smart enough to know that there isn’t anything better right now, although he is hoping for alternatives to develop over time. He would prefer to expend energy in improving possible certifications, rather than fighting over the relative value of this particular one. That’s a position that’s hard to argue with.