The Australian Securities Exchange (ASX) has automated just over half of the 68,000 test cases needed to ensure software underpinning Australia’s financial markets remains reliable and functioning as intended.
A test case is a set of actions performed on a system to determine if it satisfies software requirements and functions correctly.
The ASX is seven years into test case automation, a journey that started out with Simon Miller - ASX’s test automation manager - and a single copy of QuickTest Professional (QTP).
QTP was originally designed by Mercury Interactive, before being acquired by HP and then eventually by its current owner, Micro Focus. It is now called unified functional testing (UFT), and remains in use at the ASX.
“It was just me working on one platform using QTP,” Miller told Micro Focus’ Realize 2020 ANZ virtual summit.
“Now obviously I was doing the best job I could, but there's only so much one person can do. There was a lot of low coverage.”
Fast-forward seven years, and there's now a team of 29 people “currently involved in test automation at the ASX, working across all our domains and multiple applications,” he said.
There’s also a formal learning program in place, which was used to upskill the existing test team, and technical career pathways to offer those people progression opportunities.
Upskilling existing staff was seen as important because they had domain knowledge of ASX’s systems.
“Obviously we could have just hired all 29 test automation people and been done with it,” Miller said.
“However, we have a lot of platforms that are unique to exchange environments and that you're not going to find outside [of] there.
“A lot of our testers have been with the ASX many, many years and had a lot of domain knowledge, and that's very important to us. We wanted to make sure we didn't lose any of that.”
That also speaks to one of the aims of introducing more automated testing - to improve the reliability of critical software underpinning Australia’s financial markets, reducing the risk of bringing new features and functions to market more quickly.
“We always put our customers first, and we ensure that our systems and platforms are using the latest technologies to deliver the best focus to our customers,” Miller said.
“These systems are critical to the Australian financial market and the Australian economy in general. Each month, we have about 35 million trades just on our equities trading platform alone, so you can understand some of the numbers that we're going through each day.
“As one of our old general managers used to like to say, if the market gets halted, it not only makes the financial news but it would actually make headline news as well.
“It doesn't happen often, which is an important part of what the test team does - ensuring that we are consistent and we have the reliability - but if we lost one of these systems [and] had to halt the markets, it could potentially cost millions to the financial economy.”
Miller said software engineering and testing practices within the ASX are now tightly integrated, with test automation embedded into development practices such as the exchange’s continuous integration and continuous development (CI/CD) workflows.
“We have embedded the testing fully into the delivery pipelines and applied modern engineering practices to our testing,” Miller said.
The development and test teams used many of the same tools and worked to a single set of coding guidelines that apply “across the whole of engineering”.
“All the engineers - that's the developers and the testers - use the same coding guidelines to ensure consistency. It is all code after all,” Miller said.
To move faster, the ASX has also adopted Agile delivery and Scrum methodologies in its software development.
An automation milestone
The exchange recently surpassed 50 percent of its test cases being automated.
“We have 56 percent [automated], and that has really come together in the last couple of months,” Miller said.
“Everything is coming together - we've got the training, we've got the positions, we've got the continuous delivery.
“We have 30 years of test cases built up, so we have about 68,000 test cases, and we have over half of them automated now, which as you can imagine is great for being able to do our regression testing. I feel it is just the beginning of what we can do.”
Miller said that without test automation, applications could take months to regression test.
“On some systems where we have a lot of automation, that … is now measured in hours,” he said.
“As you can imagine, it's increased our speed to market. Because these are repeatable and we run them every sprint, we now have reliability as well and we have less defects going into production.
“The systems have to be reliable because, as I said at the start, we have millions of trades going through every single day, and the ASX brand works because it's a trusted brand.
“If we don't have the reliability, people have the option to go elsewhere. They could trade in other markets, which would be taking money directly out of the Australian economy.
“So we need to make sure that everything works and have as close to 100 percent uptime on all our platforms.”
Miller added that his team is now starting to trial behaviour-driven development (BDD) techniques for automated testing, using the Cucumber framework and tool.
“BDD is currently on trial on one project,” he said.
