Setting a Foundation for Successful Test Automation

Index

Thanks

Introduction
2.1 Test Strategy
2.2 How to Setup a Test Strategy

Designing a Test Automation Strategy
3.1 Have a Plan
3.2 Team Capacity
3.3 The Process

Creating a Culture for Test Automation Success
4.1 Communication

Developing for Test Automation
5.1 Test Automation Pyramid
5.2 Code Seams
5.3 UI Element Selectors
5.4 Tooling for Test Automation
5.5 Things to Consider in your Test Automation Project

Future-Proofing your Test Automation Efforts
6.1 Run the Tests in Parallel
6.2 Avoid test Collisions
6.3 Clean Code
6.4 Code Reusability and Maintainability

Scaling your Test Automation
7.1 Things to Consider

Measuring the Value of your Test Automation

Conclusion

Bibliography

1. Thanks

This guide was created based on the course “Setting a Foundation for Successful Test Automation” by Angie Jones.

Course available at “Testing Automation University”.

2.1 Test Strategy

Most people start an automation project by creating a POC (Proof Of Concept) when they should in the first place create a test strategy.

Strategy > Proof Of Concept.

2.2 How to Setup a Test Strategy

Define a goal and test automation strategy that meets your business needs.
Cultivate a culture within your organization.
Choose the right tools.
Optimize and scale your automation project.
Quantify the ROI (Return Over Investment) and share the values of your test automation with your organization.

3. Designing a Test Automation Strategy

3.1 Have a Plan

Before starting the automation you must need a plan, it can be defined via acceptance tests, gherkin scenarios or test cases.

3.2 Team Capacity

Make no mistake, software test automation is a software development project itself.
Since the software developers are coding new features and the manual testers are testing it, it’s up to the test automation engineer to work with the developers and testers to include the automation into the team capacity.

3.3 The Process

The test automation engineer work involves the steps below.

Writing scripts.
For your automation project you can choose a tool with a programming language or you could use one that is codeless.
It’s also important to develop standards and conventions for the future inclusion of more people in the project making it more future-proof.

Measure the maturity of your test automation project.
Depending on the automation project maturity the tests can be executed locally or in a private CI build that’s not exposed to others, then move to a separate CI job that is visible to everyone, but still does not affect the developer check-ins.
Once you have enough confidence in your automation project, add it to the development CI process.

Monitor results.
Unmonitored tests are untrustworthy tests and without your team trusting the tests they will rely less on them, leading the automation project to death.

Update results.
Ideally any test will fail due to the devs changes and therefore they can fix their code.
It will also be necessary to update the tests as the application behavior changes, if new features are added you will also need to write new tests to be implemented and maintained.

“Manual testers are also a viable option for triaging automation failures. Especially, ones who have an interest in contributing to the automation code base. Reviewing test failures, debugging the automation code, and updating the code when needed are great introductory opportunities into test automation.”
– Angie Jones

PS: Checking-in code means to upload code to the main branch repository so that its administrator can review the code and finally update the project version.

4. Creating a Culture for Test Automation Success

4.1 Communication

Test automation is most successful when there’s collaboration amongst the team members, the product owners, business analysts, developers, testers and automation engineers.

The best approach to convince the team about test automation is to communicate the goal, strategy and opportunities for collaboration.

The product team can provide information on what scenarios are best to automate.
The automation engineers and the testers could assist each other since automated tests do not replace manual testing, the testers are necessary to explore the application being the ones that will discover your most costly bugs.

5. Developing for Test Automation

5.1 Test Automation Pyramid

Unit Tests.

Fastest implementation.
Unit tests are the closest tests to the production code.
Can pinpoint bugs in the exact function it happened.
Largest number of automated tests should be here.
Should be inside the project and be written in the same programming language as the project.

Service Tests.

Test the functionality of the code without an UI.
API tests, need to be able to execute HTTP calls and read through responses.
Second largest number of automated tests should be here.

UI Tests.

Longer to write and to execute.
Most expensive.
Least number of automated tests should be here.
Need a navigation library so you can interact with the HTML elements.

5.2 Code Seams

Code seams are shortcuts in the application which make automating easier.

For example, imagine you have to automate the test of buying a product, through the UI it would be necessary three steps:

Select the product.
Add the product to the cart.
Click the buy button.

Given that you just want to test the step 3 feature, and step 1 and 2 happens to have a bug leading the whole test to break making it flaky.
With a code seams you could bypass the UI, send quick HTTP requests, access the step 3 directly and then test only the desired feature.

PS: Additional code seams can be provided by developers so you can access the UI through the service layer.

5.3 UI Element Selectors

In the UI level the test automation project interacts with HTML elements.
It’s important that the developers add identifiers to these elements such as ID and name attributes.
Remember and guide the front-end developers to add these attributes.

5.4 Tooling for Test Automation

The minimum needed for an automation project is an interaction tool (Selenium, Appium) and a validation tool (TestNG, JUnit).

Make sure the tools you are using support the browsers and devices you need to run your automation.

5.5 Things to Consider in your Test Automation Project

You can add a reporting library that can show screenshots and/or videos.
You can integrate in your automation project gherkin scenario files if your team is practicing BDD (Behavior Driven Development).
You can also add tools to add your automation project into a CI.

6. Future-Proofing your Test Automation Efforts

6.1 Run the Tests in Parallel

Design all of your tests to be independent.
Avoid modifying test data that another test may rely on.

6.2 Avoid test Collisions

If one test tries to change its data while another test tries to use it (for example when a test changes a password and another test tries to do login) the second test will fail.
To avoid it, make the tests that need to change its data to be the only test using it.

6.3 Clean Code

Avoid these:

Excessive code duplication.
Long classes and methods.
Inefficient waiting within tests.

6.4 Code Reusability and Maintainability

Use design patterns in your project:

Page object model.
Screenplay.
Fluent.
Builder.
Singleton.
Factory.
Facade.

7. Scaling your Test Automation

Take into account that multiple environments have different information such as:
IP addresses, URL, login credentials and database information.
For this motive the best is to extract this sort of information from your automation project and use properties files.
Containerization helps to mitigate some of this burden.

7.1 Things to Consider

Multiple Browsers (Choose which one according to your business needs).
Multiple Devices (Android, iOS).

8. Measuring the Value of your Test Automation

Identify expectations early and communicate it to the team.
Measure the amount of time the regression test takes to execute.
Fast and frequent feedback, eventually the automation should be able to run every time a dev checks-in code, delivering fast feedback.
Monitor team level of trust on the automated tests, ask them in the retros.
Scalability is the ability to scale automated tests to run across different environments, browsers and devices.

9. Conclusion

A test automation project is not a side project, it’s a software project development.
Things to consider before starting your test automation initiative:

Define a goal and strategy.
Culture, ensure that your company supports this strategy.
Choose the proper tools to automate.
Scale your test automation beyond running it locally.
Set realistic expectations over ROI (Return On Investment).

The end.