You don’t need Test Automation. Here’s why.

There are three major reasons why clients opt for Automation Testing. AT speeds up the project progress, it saves costs and gives more accurate results as the human factor is eliminated. Let’s consider each of these points in more detail and see if they are genuinely true things.

Suppose you have just started your software development project. The core idea is relatively easy to implement, yet the devil is always in the details. This means changes are inevitable and some features may change on the fly. This is a BAD BAD thing if you’ve hired an Automation engineer instead of a manual tester from the start. Where a human can adapt instantly and quickly check the modified feature, the Automation engineer has to rewrite costly autotests first. Thus, right from the start, we’ve bumped into the top challenge of Automation Testing, which is test maintenance. And it eats up a great deal of the project time. So nope, Automation Testing won’t do you any good here.

What if your project is going full blast and the system has grown so many features the manual testers can’t keep up with the development team? Yes, this is the moment you should introduce Automation Testing, but test maintenance will go nowhere. It is the problem to stay and it needs your project hours to be fixed.

You may have heard about the cases where Automation Testing did miracles to the project budgeting. They may be true, but in the nutshell, Automation Testing is a freakingly costly thing. Salary stats vary from country to country but be ready to pay twice as much for an Automation engineer than for a manual tester.

QA Automation engineers usually can customize open-source test automation software such as Selenium, Serenity, Appium (see more here) for your project needs. However, customization implies great coding skills, so if your talent won’t be able to tame that beast of Selenium, for example, to serve your needs, licensing costs are inevitable.

Last but not least, with open-source frameworks reporting can be an issue. The paid alternatives provide as customized reports as you want, while open-source ones just give you the core info, which sometimes may not be enough for the clients or investors. Hence, add up licensing costs.

It is indeed when it’s done right.

However, if your Automation engineer does not know the ins and outs of the system under test yet, it will negatively affect the testing results. The truth is, the quality of test automation depends on the quality of autotests. Say, an engineer who is not yet deeply immersed in the project, may just not be aware of the changes that were introduced. He/she would keep racking his/her brain over the weird test results while the module under test might have been simply deleted. Without the profound project knowledge, the engineer would spend hours fixing the test case and wasting your time.

Second, if your functionality undergoes frequent changes, test automation will be a drastic waste of money. Tests take time to write, so if you make changes it means corresponding changes should be made into your test cases or no one will need that pile of ready autotests.

Well, it’s sort of obvious from the above paragraphs.

1. To save money.
2. To test new functionality quickly.
3. To avoid ambiguity caused by poor test maintenance.

When done properly, manual testing brings great results.

But what do we mean by done properly? Here’s a simple cheatsheet for you to check your existing manual testing practices against.

Elaboration phase

1. QA lead joins to study the Specifications and prepare a Test Plan and Test Coverage Matrix.
2. The Test Plan and Test Coverage Matrix (test scope, platforms, documentation, etc.) are then sent to the client for review.

Zero Sprint

1. This is a Pre-launch phase during which the QA team studies the Specifications Requirements in more detail and gets to know the project.
2. Preparation of user stories and test cases follows.
3. First smoke tests launched.

Active Development Phase

1. After each sprint (and sometimes in the middle of the sprint), regression tests are performed.
2. QA team also does smoke tests for each new build to identify critical issues.
3. Acceptance checklists preparation.
4. Known Issues Weekly Reports are generated to keep the client aware of minor, trivial and major issues as well as blockers and correct the development deadlines accordingly.
5. Ad-hoc or Exploratory testing is performed only if specifically mentioned by the client. This is done so to have all the tests properly documented in order to give the client a robust test documentation after the QA project is complete.
   
   During a QA project, a QA engineer creates a separate subtask which is a describtion of the feature task in the form of a set of tests. This guarantees no test or bug goes unreported.

Active Testing Phase

1. System testing against the acceptance checklists.
2. Client testing and user testing.

Project Closure

After all the tests are done, the client has a full test documentation package including checklists, test cases, test reports, KIWR, and acceptance checklists.

However good the manual testers might be, Test automation is a must in several cases.

• The bigger your system grows, the more acute the need for test automation. So, here goes #1 on our list - large long-lasting projects. By long-lasting, we mean 6 months and more.
• API, protocols, etc. where manual testing is ineffective.
• Mission-critical systems are always tested using test automation tools. Those are the systems pertaining to space, aircraft, healthcare, logistics, and other industries where the cost of a mistake is too high.
• Systems to undergo industry certifications such as banking and finance software must be autotested, too.

Even though the IT industry is growing by leaps and bounds, manual testing is not going anywhere anytime soon. At the same time, test automation also must be a part of your testing strategy, when appropriate (long-lasting projects, mission-critical systems, etc.).

The best option in our opinion is to combine manual and Automation Testing for the best results. Start small from manual testing and, as your system is growing bigger, automate smoke and regression testing of the critical mostly used features that do not undergo regular changes. Mind that new features can be properly tested only by manual testers as automation testing is typically limited to regression capabilities.

Find your balance between Manual and Automation Testing and this will bring you costs reduction, reasonable speed, and accuracy all at the same time.

Hesitant as to which type of testing suits your project best? Symfa has extensive experience in both manual and automated testing. Contact us and we’ll be happy to share with you our insights on the tropic. Regardless of your project complexity, we’ll help you find out the most cost-effective solution that will bring top testing results.