Test automation — using automation tools to execute test case suites — delivers numerous benefits, including greater time- and cost-efficiency, the ability to run tests unattended/overnight, and a lower risk of integration and production issues. Particularly well suited to automation are test cases that are:

  • New or modified functionalities
  • Business critical
  • Repetitious
  • Tedious for humans to perform
  • Performance sensitive
  • Time consuming

It’s important to recognize that automation can’t eliminate all manual testing because automation is for testing functionality. You still want users doing hands-on testing to ensure the usability of the application. 

If your organization is considering automating parts of your testing processes, here are three things to keep in mind.

1. Cost and time requirements will be lower than you think.

Many teams don’t implement test automation because they believe that it takes too much time or too many resources. While this may have been true in the earlier days of test automation, it is no longer the case with modern tools and cloud services. For example, most tools can record user interactions and then allow developers and testers to modify the results for more dynamic testing. Also, most automated build tools can now incorporate many test engines into the build process, catching issues before they are deployed, and can impact live production use.

2. There are no static inputs in the real world.

Teams that do automation testing often use static, hard-coded parameters rather than dynamic parameters. This not only misses mimicking real-world use cases; it also will not properly reflect performance and scalability metrics where caching is in use. Dynamic parameters can either be randomly generated using standard scripting languages or driven from prepared input files. The parameters also need to be as realistic as possible. The use of nonsensical values to populate text fields and simple sequential numbers for complex number fields such as phone or amount can miss validating even the minimal edge cases

It is also important to have in place a procedure to reset databases to pre-test conditions for consistent re-testing.

3. Know when to run the automated tests.

In the Test-Driven Development (TDD) approach, the tests are written before the code. Even if your development team doesn’t follow TDD, writing tests as soon as the interfaces are created and callable can save hours of time spent performing manual functional tests. The amount of testing necessary to get from the first stubbed interfaces to ready-for-deployment into an integration environment is almost always underestimated. Not writing the tests as early as possible consistently results in either increased effort during development, manually testing functionality as the code changes (versus a button click or one-line command to run the automated test), or increased time during quality assurance debugging all of the scenarios that were insufficiently tested during development

Even if you balk at automating functional testing, testing before turning over to QA will save many hours spent in the writing of defect reports, triage reviews, debugging, and re-testing. Most true DevOps approaches include testing with every build-and-deploy cycle.

Conclusion: The bottom-line is about the bottom line.

The time spent developing and maintaining test automation will deliver a positive ROI by reducing the number of production issues and shortening QA cycles. The time it takes to realize that ROI will vary based on complexity and technical culture, though it is often much sooner than anticipated even with the best application teams.

As with most cases of process automation, automating testing is neither a silver bullet nor a one-size-fits-all solution. The value of real user testing is still valid and necessary, as there will always be use cases that are not anticipated by designers, developers, and testers. By taking the time to determine all use cases that can be automated, selecting the tool that best meets your organization’s needs, and leveraging best practices like using dynamic parameters and testing as early as possible, you stand the best chance of improving productivity, shortening QA cycles, building customer trust, and achieving positive ROI.

Like what you see?

Paul Lee

Senior Director Scott Nelson has over 20 years of experience working with the full delivery life cycle of solutions for system integration, process automation, and human workflow applications.