How a MedTech company automated regression testing on their flagship blood analyzer which was considered “non-automatable” with TestResults.io

Challenge

  • Boold analyzer control software can't be tested automated
  • Manual testing is too time-consuming, and resources are spare
  • Testing coverage is not what it could have been

Solution

  • automated human-like software test automation (black box testing)
  • FDA-compliant without additional effort
  • No need to re-validate the automation platform with future updates

Result

  • 120 e2e test cases automated in 102 days. On average, less than 7h/test case
  • 3 software versions covered with each automated test case - previous, current, and upcoming
  • Ideal test coverage while freeing up tester resources

The Company

Summary: A major American manufacturer of diagnostic equipment for blood testing. Bringing diagnostics to the next level with their blood analyzer robots.

Size: 6'000+ employees

Products: Diagnostic healthcare manufacturer

HQ: USA

The Challenge

TECHNOLOGY THAT CAN'T BE TESTED AUTOMATED & LIMITED SOFTWARE TESTING RESOURCES

A major American manufacturer of diagnostic health care products that sell worldwide struggled with software regression testing for its blood analyzer robots, its flagship and cash cow product. The company could not automate software testing because of the technology used for controlling the software.

A limited number of testers performed regression testing, so the analyzer was not tested as much as it could have been—a scenario many companies can relate to.

One of the challenges of automation was that the user interface technology used virtualized elements and remote sharing. The widely adopted approach in automation tools to test based on locators  (XPaths, IDs, other identifiers) cannot be applied in such a scenario. 

Therefore, they were not blocked from reducing the human effort while testing resources were sparse.

The Solution

AUTOMATED & COMPLIANT HUMAN-LIKE SOFTWARE TESTING (BLACK BOX TESTING)

As TestResults.io interacts with the applications under test like a user would – pure black box testing without access to the code – they could automate the entire regression testing and free up their testers to do more valuable exploratory testing.

TestResults.io is one of the only tools available that was made explicitly for the MedTech industry and its needs. Out-of-the-box, it has zero levels of intrusion, traceability, and revision-based reporting. No additional effort is needed to keep testing FDA-compliant.

Besides that, they don't need to validate TestResults.io with every update, as you usually would have to do with automation solutions, to stay compliant.

TestResults.io offers the so-called Frozen Solution for regulated industries,where they get all updates without having to revalidate the automation tool. An unexpected additional benefit that highlights the difference between a test automation solution and a test automation solution built for MedTech.

The Result

120 E2E TEST CASES ON 3 SOFTWARE VERSIONS AUTOMATED WITHIN 102 DAYS

The initial automation of all their test cases took less time than expected. People usually hesitate to start an automation project because there are insufficient resources, and automation is a huge pile of work. However, they were able to automate 120 complete end-to-end test cases for three different software versions in 102 days.

With a bit less than 7 hours of engineering time per test case, they do not just have their automated regression testing; each test case covers their previous, current, and upcoming software version—which are all in the field, used by customers.

Not being able to automate your software testing takes a lot of time from your testers, which means low test coverage. This means a huge risk for your product, especially in a field where a software error may end someone's life.

TestResults.io doesn't rely on the underlying technology of your applications and can test medical devices like this blood analyzer robot, a platform for cataract (phaco), glaucoma, and vitreo retinal surgery (eye surgery), or any other medical device with a user interface; you can automate any software.

Side benefit: Less sweaty palms at your next audit.

Within weeks, we uncovered bugs in the newer version that we hadn't found in previous regression runs. We are running ten times more regression tests for every release without user interaction. We aim to increase that even more.

- Development Lead
This is some text inside of a div block.

Automate your software testing of non-automatable applications

TestResults.io helps you comply with the MedTech regulations for your medical devices, and it's set up fast and never again software testing resource problems.