...When Testing Complex Software Systems

This article is relevant for you if you are involved in the quality assurance of complex software systems and if your top priority is maximizing customer satisfaction and the value the software delivers.

One of our customer in the MedTech sector managed to save 700+ h per year! Since lives rely on MedTech software, it is subject to strict regulations. And all the regulations pose challenges and don't always support efficiency.

Our client was able to overcome them!

To make testing complex systems more efficient, precise, and reliable, they didn't just switch to TestResults.io as their test automation platform—they also introduced some smart additional methods on their test automation projects.

We'll reveal three of the methods they use!

You can apply these methods in non-regulated environments just as effectively when testing complex systems to save time, simplify processes, and ultimately deliver software that impresses customers.

Steal these methods and simplify your own testing of complex systems ⬇️

1. Applying IQOQ for Non-Regulated Software

IQOQ is a common practice for medical devices and other regulated industries, and it helps when complex software systems are developed in your non-regulated field!

In regulated environments like health tech and medical devices, Installation Qualification (IQ) and Operational Qualification (OQ) are essential to ensure systems perform correctly in their specific environments.

Since we will go into more detail about IQ and OQ in this tip, it's good to know the definitions:

Definitions

Installation Qualification (IQ) ensures that an instrument or device, subsystems, and ancillary systems are installed and configured according to the manufacturer's specifications or installation checklist.

Operational Qualification (OQ) is performed after the successful completion of IQ protocols.

OQ aims to verify that the device's performance meets the user requirements within the manufacturer-specified operating ranges. Why this practice is also helpful for you:

Similar situations in non-regulated environments

When deploying a complex software system in a new environment (especially at a customer's site), performing a few sanity checks is crucial to ensure that the software is installed correctly and all dependencies are in place.

The Sanity Checks

• You test some essential features of the software.
• You test features you know depend on third-party dependencies and interfaces.
• ...in the (customer) system where you have just installed.

Standardize and document these tests

When done frequently, and since automation is usually already in place for testing during development, why not automate these IQ / OQ checks as well?

Provide self-contained IQ/OQ-Check software that executes these automated tests and gives you confidence that the software will run as intended.

Automating these tests gives you confidence that the software will run as intended and reduces the effort for service or operations engineers when installing the software system in a new environment.

It's Your winner formula!

When it's already standard practice for medical devices and other regulated industries, applying it in your non-regulated field is a no-brainer when complex software systems are developed.

2. Amaze Your Team, Your Boss, and Your Friends in Dev with the Best Test Report

You already know how vital good test reports are.

The game is simple: clear test reports -> easier debugging -> better software quality -> you've done a great job! 🏆

How to get the most out of your test report

Capture and report an image of each element when the automated test interacts with it!

This Helps with Debugging

With the image, it's immediately clear:

• What text was really entered in that textbox.
• Which value was selected in a dropdown list.
• What the state of the button was when it was clicked.

Save yourself an extra step

When done right, these reports serve as the basis for documentation such as user manuals.

You feed two birds with one scone!

1. The Error Handling in Test Automation for Champs

99% of people in QA don't remember what they automated a year ago.

Don't bother…

It's not because they're forgetful but because it's perfectly human not to keep all the small details of a test case in mind for an entire year.

You're so active and accomplishing so much daily that if you could remember it all, one might argue you hadn't done much.

So, don't worry about remembering all the details of how you automated the test cases.

Just ensure excellent error handling so that remembering details isn't necessary.

Already think about maintenance and debugging during the development of the automated tests!

4 Magic Steps

If you do this now, you'll be chilling when debugging a couple of months or years from now:

1. Capture and Communicate Mismatched Values

In case of a failed verification, always return the actual value that didn't match the expected value.

Report something like this:

"The selected value on (element) was (actualValue) instead of the expected value (value)."

2. Log Fix Suggestions for Missing Preconditions or Test Data

Report / Log fix suggestion when incorrect precondition or test data is detected.

Example:

Your test relies on a DB script that creates a record. If this record is not found during test automation, include a message that the DB script needs to be run before the test.

3. Implement Retries for Network-Related Issues

Perform retries for specific issues, such as network problems.

This will help improve the stability of your tests.

4. Dynamically Handle Unexpected Popups

Dynamically monitor for unexpected popups and handle them.

Example:

Close an update notification that appears on top of your test system.