Open Source Subnet An independent Open Source community View more

Revisiting the shadow quality of APIs

Examining the disconnect between developers and testers.

I had the happy fortune to be at StarWEST a number of weeks ago, which together with StarEAST is one of the major quality and testing conferences to take place in the U.S. every year. As always, it was a fantastic opportunity to take the pulse of the testing community – beginners, practitioners and gurus gathering to share their new and old knowledge about testing methods, technologies and challenges.

Not surprisingly, my session at the conference was related to web APIs; I spoke on their transformation from internal technical asset to public business asset and what that means for corresponding testing and the quality efforts. From a quality perspective, this shift is not be taken lightly - from being hosted and consumed internally in a controlled environment, these APIs are now exposed to external users with unpredictable usage. Obviously, this gives testers and quality professionals more to consider when devising their testing strategies. No longer is it enough to make sure that your API returns the correct results; it also has to “look good” and be reliable – and do so in a competitive way, as there are many similar APIs available to the consumer.

I won't dive into the fascinating world of API quality here, as I've touched on it in several of my earlier blog posts, but it basically comes down to three major concerns:

  1. Usability – Make sure that your API is as appealing as possible for its target audience. This includes focusing on documentation, metadata, nomenclature and technology alignment.
  2. Reliability – Make sure that your API is reliable and performing well enough to be trusted as a key component in someone else's business. This includes ensuring security, transparency, performance and scalability.
  3. Functionality – Make sure that your API actually works as it is intended to, which includes testing expected results (data, formatting, consistency), error messages and standards compliance.

Looking ahead, I will soon attend an API-focused conference – API Strategy & Practice in San Francisco. But whereas StarWEST had almost no APIcoverage (my session was the only session related to APIs of around 70 sessions), APIStrat is exclusively about APIs – both from a business and technical point of view. Adversely though… only one session is focused on API quality.

It seems the intersection between the world of APIs and world of testing is pretty small. Doesn’t the API crowd care about quality in their products? Do testers ignore APIs in their test plans and strategies? Or could it be that the people attending these two conferences come from slightly different worlds? Are the testers and QA professionals at StarWEST from a corporate/legacy world where testing is most often part of a well-established and embraced process? At APIStrat, the API geeks (for the lack for a better word), from startups or “web 2.0 companies,” in smaller teams, embracing agile approaches and breaking new ground – in other words, organizations not “mature enough” to fully embrace quality and testing?

Obviously, that is an over-generalization – there are plenty of testers interested in APIs at StarWEST (as was evident in my session), just as there are project and product managers from large process-heavy corporations at API conferences (as was evident at APIStrat earlier this year in New York). But I also think there is actually more of an intersection between quality and APIs than what the above observations might indicate; a “shadow quality” movement born from the inherent need within agile development to make testing an integral part of the development process. Developers in the new API world have been embracing unit-tests, test-driven development and BDD from the day they left school (not too long ago), and testing is an integral part of their work when they plan out iterations and tasks. They really care about quality, and don’t mind writing complex test scripts that can be run for every continuous build and deploy of their solution. Just look at the abundance of open source test-related frameworks and toolkits out there; these all come from either quality-aware developers or code-savvy testers who strive to make testing easy and natural. In other words, testing has taken a deep (and much-needed) plunge into the development soup.

If you’ve followed this blog, you know that I don't think you can really compare testing with “checking”; there is still a big difference between developers, who want to make sure things work ("checking"), and testers, who will go to any length to try to break things (“testing”). But there is also a hugely increasing group of developers who really care about quality. They are often found embracing new technologies (like web APIs) and applying their desire and commitment to high-quality deliverables in their everyday work.

Ultimately – shadow quality or not – comprehensive testing of APIs needs to be an integral part of an API implementation for it to be successful in the long run – and if that is primarily done by developers with a quality mindset writing tests, I’m all for it. The winner in the end is the API user anyway.

From CSO: 7 security mistakes people make with their mobile device
Join the discussion
Be the first to comment on this article. Our Commenting Policies