Everybody Loves Raymond

Especially since he’s the only one in the team who knows something about automation software testing…
Isn’t that the case? There’s that only one person, who knows something about the one thing the whole group wants – how to automate more of their manual tests.
And that’s never enough of course. So many testing groups are trying to get started with automation testing, just to find themselves failing. Sometimes failing to even kick-start the project, sometimes starting with great expectations, just to abandon the project a few months later, with no real results, and in other cases somehow completing that big automation project, just to find that they have created a monster that needs to be constantly fed (meaning, it costs a lot, requires many expensive experts, and that’s on top of the hardware and everything around it).
So what are the options? Unless you are a very big company, with unlimited resources, and can get the budget for a really complicated and long project? If you’re not there, you still need options to automate more of your software testing.

Reality fact check

The idea with software automation, is to be able to test more, to test fast, and to save time and money (on testers). Automating your tests, or part of them, enables you to direct your manual testing efforts to the things that automation will not do for you (UI, design, look and feel, exploratory testing, etc.). Together, automation and manual testing, when done right, will bring the best results (which means – better products).

Consider starting your process by answering a few questions:

  • What part of my tests can be automated?
  • What part of those tests, that can be automated, is important to automate (meaning that automating them will save considerable time and money, and will yield better results, or at least the same level of results).
    Do you have the resources (budget, people, time, tools) to automate these selected tests?
  • What will be the required maintenance (budget, people, time, tools) for these tests, on an ongoing basis?
  • Considering the scale of your software projects, and specifically your testing projects, is it all worth it?

Once you’ve decided that automating your tests, or part of them, is the right thing for you, you can start planning how you will do it.

Automating your Tests

That’s of course the main challenge. 99% of testing groups will find that the answer to that last question is Yes (yes, we want to automate as much as we can…), but are not sure where or how to do it. Where to start.
You’ve probably heard or even saw automation testing, running somewhere, but didn’t see the process, just the outcome. And getting it done for your tests seems too big of a challenge, involving too many “thinking” tasks and too many “technical” tasks.

We suggest approaching this challenge differently.
Assuming that your project has some minimal set of tests, which you must run each time before releasing a new version, let’s name them your Sanity tests. In most cases, these can be between 5-10 tests, that test the most important parts of your software. The parts that you have to know for sure, are (still) working properly, before you release the new version.

So, take these 5-10 tests (the Sanity tests), and only these, and answer two questions:
1. Can I automate these tests?
2. Can I trust these automated tests?

The first question is answered easily, because you can choose any automation tool, Selenium for example, record these – still manual – tests, and let Selenium turn them into automation test scripts.

Once that is done and answered, move to the next question, which is trickier. It takes several testing cycles to get the probability, and therefore confidence, you’re looking for. You will know if you can trust these automation tests, to replace the same tests that were done manually beforehand. So remember to run these tests both manually and with their automated versions, in the first cycles, so you’ll have a comparison.

Once you’re done and satisfied with this first entry to automation testing, with the minimal, but important, sanity testing, you’re ready to get into the automated world of testing.
This means automating other parts of your tests, using your selected automation tool to do more for you, not only in quantity (more tests, running the same tests more times), but also in depth and quality of testing (sophisticated tests, code coverage, scheduling, repetition, removing unnecessary tests, and more).

We’ll be discussing automation testing more in coming posts. Stay tuned.