Test automation, the process of automating tests, is becoming more and more relevant these days. Especially with agile methods like Scrum, Kanban, XP etc. where methods like TDD and BDD are in place.
Despite the safety net automated tests provide there is one big difference between an executed automated test and a manual test (whether scripted, ad hoc or exploratory).
I once got the question, why I should test a feature/story manually again when there are already automated tests for it. The answer to it is quite obvious in my opinion. Automated tests only cover functionality and behaviour we know about. We have taken an expected outcome and check that the application still works as we think it should. The expected outcome is usually derived from requirements. But there is no way to deal with unpredictability, uncertainty and unknowns. And who actually tells us that the requirements itself are correct?
So rather than relying on automated tests it is good to explore the application under test, learn about it and feed back the learnings to possibly new automated tests.
In his blog Testing vs. Checking Michael Bolton describes the difference between a test and a check which also explains why automated tests on it’s own are not enough.
So how can we explore the application and question existing requirements? Dan Ashby has got a really good blog about unclear requirements called Who needs questions – let’s just test.
To sum it up Test Automation is an essential part of agile methodologies. Nevertheless you can only automate what you know. Testing a piece of software involves exploration and going beyond your knowledge of requirements and leaving the happy path.
