Entire test automation for central fraud-detection broken by a technology change and how a major Swiss bank ensured this will never happen again by applying TestResults

The Challenge

ENTIRE TEST AUTOMATION WAS BROKEN WITH THE NEW RELEASE OF THEIR PEGA FRAUD-DETECTION SOFTWARE

One of the major Swiss banks, internationally active, uses Pega, a well-known fraud detection software. To enhance the software, the tool provider changed the underlying technology while the user interface remained the same – we assume to make sure no retraining for the employees was required.

This new release broke the entire test automation they had in place. Instead of repairing the whole test automation, they decided to change the tooling to ensure this would never happen again. The switched to a test automation platform for their critical systems which does not depend on the technology of the software under test—especially if it’s not their own software.

The Solution

AUTOMATED HUMAN-LIKE END-TO-END BLACK BOX TESTING

The only way to ensure such updates don’t break the entire test automation is to use a test automation solution that doesn’t rely on the code of the software under test: Human-like black box testing, but automated. In other words, TestResults.

As it is so simple and easy to use, they added their apprentices to the software test automation project—with huge success. The flexibility in resources was a great side effect of switching to TestResults.

TestResults gave them extra confidence as it is built for regulated environments. It supports you in being compliant with the financial market authority and the enterprise requirements in these regulated surroundings.

The Result

99% LESS FLAKINESS THAN WHAT THEY WERE USED TO

While automating the test cases, they were surprised at how much faster they were, even without any experience in TestResults. The first two test cases, each around 50 steps with 20 different screens, were automated in 3 days—much faster than they were used to. In fact, they were trained experts on their previous tool. ‍ Besides being much faster in test automation than before, they reported having reduced their flakiness by over 99%, as no asynchronous processes are running in the background that were previously needed. TestResults Screen Monitoring and AI wait until the site is loaded and usable. Before, almost 40% of their test runs were flaky, which required a lot of additional resources, which is, by definition, far from the idea of automation.