We’ve covered extensively what to look for in a test case software management platform, including:
- Ease of use
- Upgrade frequency
- Add-on compatibility
But today, I wanted to explore what not to look for – or more specifically – what marketing gimmicks and “promises” you should take with a grain on salt when comparing one test management system with another.
So without further ado…
Test Management Pitfall 1: Feature Comparison Charts
This one might surprise you. After all, feature comparison charts are incredibly popular and “useful” tools for comparing 2 or more related products.
But don’t be fooled.
With the right approach, a feature chart can position a Toyota Camry higher than a Rolls Royce. If the Rolls Royce doesn’t have cup holders, a detachable CD player, or seat warmers, it must be an inferior vehicle, right?
Remember that feature comparison charts are not a very accurate reflection of your needs and wants as a user. The manufacturer simply takes stock of all the bells and whistles of his product and then shows you how the competitors are missing those bells and whistles.
Test Management Pitfall 2: Lack of Specifics
In the US, if a food manufacturer adds a dash of pepper to the recipe, it can claim that its product is made from all-natural ingredients.
The same trend happens in test case management software. Don’t be impressed by generic – and often misleading – statements like “Integration with Jira.” The casual reader might believe that this platform truly integrates with Jira and offers real-time synchronization – when in fact, it’s not really offered or making the 2 work requires a lot of back-end customization (performed by the user of course).
- integration with
- synchronization with
- compatible with
… these are loosely defined ideas that vary considerably from manufacturer to manufacturer. Be sure to read between the lines.
Test Management Pitfall 3: Hidden Prices & Fees
Barbie doll marketing is a business concept in which you lure users in with a low price (i.e. the Barbie doll) and then hit them with expensive add-ons (the house, the clothes, the shoes, etc.).
You see this with modern printers as well. The machine is cheap – the cartridges are killer.
Unfortunately, test case management software suffers from the same trend. The platform is reasonably priced – in fact – a bargain. But the add-ons will get you. The developer will pretend like these add-ons are optional, but if you need true integration or recording capabilities or debugging – these add-ons are most certainly not optional for you.
Also beware of switching costs – the money you have to dole out when moving from your current platform to a new one. The software might be affordable, but with installation, on-site maintenance, upgrades (now and later), storage, technical support – the price can quickly skyrocket.
Test Management Pitfall 4: Superlatives – Best, Fastest, Most
This is another marketing favorite in which test management developers try to lure you in with hyperbolic statements about performance and features. There are 2 problems with this approach:
- Best what? Fastest what? And more important, is this a feature that truly makes a difference? Or is it just a useless column in a bloated comparison chart?
- Who is the ultimate judge of best or fastest? These are just buzzwords. Unsupported claims. After all, how many cafes legally claim to have the “best coffee in town”? They can’t all be correct, can they?
The point is, you need to decide what is “best” for you. Each project is unique. Not even Testuff can be first in class in every single category (but that doesn’t stop us from trying).
Test Management Pitfall 5: Training, Manuals, and Learning Curves
This one is a no-brainer, but I’m constantly amazed by how many testers overlook the importance of start-up times and training.
Avoid test case management systems with long learning curves. In addition to spending more time and money on training, you’ll also sacrifice quality due to higher error rates.
A more expensive platform that requires minimal training may be the cheaper option – especially when you factor in upgrades and any training you’ll have to provide to future staff members.
Base Your Purchasing Decisions on Current & Future Needs
Just so we’re clear, not all marketing claims are inherently bad. Software test management developers want to position their products in the best possible light. And they’ll use whatever tools are in their arsenal.
So when you see semi-truths or outlandish claims or poorly defined terminology, this doesn’t mean you should run for the hills. It just means that you need to do a little more research so you understand the ins and outs of that particular platform.
The above is just a partial list of things to look out for. I’d be interested in hearing some of your own lists. Feel free to comment down below.