Greetings, fellow testers.
At a recent tech talk on automated testing and building, the statement was made that "not automating tests is like stealing from the company."
The speaker’s intent was to stress the importance of what he believes to be a “best practice.” The focus of the talk was to encourage using unit tests and test-driven-development techniques; automating integration, functional and user interface tests; and including them at various points in the product’s build process.
These are considered good, productive, time-saving techniques and so it should be obvious that not using them would be bad, unproductive, and wasteful — right?
Yes, of course.
No, don’t be ridiculous.
Of course, we want to be productive and, over the long run, save time and money. Reducing the number of manual steps, letting computers do our menial tasks makes sense. Our code, integration of components, and user interfaces all need to be tested, and where reasonable let’s automate those tests.
But let us not go overboard.
First of all, from the standpoint of user interface testing, not every test should be automated. Layout, color, and visual clues are best tested by eye. There are ways to use automation to make your manual process easier and faster – Test Studio’s Fast Forward, for example – but in most cases fully-automating a visual inspection test just isn’t reasonable or worthwhile. As Kobi Halperin put it on Twitter, "unproductive automation can be an even greater thief than no automation."
There are plenty of other reasons that a company may decide not to automate tests of one sort or another.
One that springs immediately to mind is the development team that builds dozens of “one-off” apps every year. Each of their products is for a specific, short-term event, and then will never be used again. The development cycle is extremely short, and the customers are more concerned with speed of delivery than overall quality – as long as it can be used to order merchandise for the event, it’s ok. Time invested in automation could result in a missed deadline, which simply cannot be tolerated.
Granted, this is somewhat of an extreme situation, but that’s the point – every company, every development team, every project has its own priorities and decisions need to be made intelligently.
Test automation, like any other practice or methodology, isn’t “magic sauce” to be mindlessly poured over everything; it should be used where appropriate. Dale Emery recently wrote that "A good automated test will satisfy (1) at least one good reason to be a test and (2) at least one good reason to be automated." What criteria do you use to determine when to automate? Drop a comment below or join the conversation over on Twitter.