Test automation for better software, customer service

"It used to take four people five days to test the software every month," says Kevin Bingham, vice president of IT at Arizona Federal Credit Union. "Now, we can do it in six hours."

Bingham was explaining how Arizona Federal got an ROI of five months by implementing test automation and a structured process using Compuware's TestPartner. But that's just the measurable, quantifiable benefits. The real payoff, he points out, has been in the IT department improving the credit union's ability to do its real job: Paying attention to making its members happy.

Arizona Federal has 750 employees and manages over US$1.9 billion in assets from 232,000 member accounts in 27 branch locations. Like most credit unions, Arizona Federal relies on a vendor's software package to manage day-to-day financial transactions. The suite of applications helps them do everything from creating a new account to preparing loan applications.

Most of the applications in Fiserv Telenavigator are updated quarterly, and the credit union has always tested each release thoroughly before deployment. Doing so provided an opportunity to resolve issues with the vendor before the software went into production.

But the testing was disruptive and time consuming, especially since it was done by Arizona Federal staff members who were not IT specialists. "The people who did the testing have day jobs," explains Bingham. For those staff, testing the software was an additional job requirement, something that pulled them away from smiling at a customer across a desk or from crunching accounting numbers. As a result, it took 60 to 90 days to test a new version of Telenavigator; if the release arrived on January 1, it'd be deployed in March. And a new release comes out every quarter.

Plus, the line-of-business employees weren't QA specialists, so the IT department had no way to validate the testing methodology. While the accounting and finance users had the advantage of knowing how they used the software every day, it would never occur to them to check for code coverage-much less to be familiar with the term.

Clearly, there was room for process improvement.

Bingham had used test automation tools in a previous role, so when he came to Arizona Federal, he suggested a similar approach and began shopping for software to create maintainable and reusable test scenarios. Price was an issue, he explained, because the credit union is not-for-profit. Whatever they chose had to clearly add member benefits and improve the organization's ability to serve those members.

Compuware's TestPartner fit the bill. At the credit union's request, Compuware prototyped simple scenarios for mainframe, Web and Windows applications. The functional prototype demonstrated that the process would work. The software acquisition was made in June 2007, along with a week of training. They've been using TestPartner since last August, and, says Bingham, it's already made a big difference.

Arizona Federal dedicated one person for QA duties full-time; this wasn't a new hire, but a techie coming out of the IT department who already had a relationship with the non-IT staff. "There needs to be a dedicated resource," says Bingham. "Someone has to be focused on testing at all times." The QA guy-his name is Jason-worked with the line of business people to create scripts that capture how the software is used. Those scripts are run against each iteration of the vendor's software to identify problems. Moreover, the same scripts can be run every time to ensure that everything is tested every time.