At DiUS we’re passionate about finding better ways to deliver software, and we believe automated testing offers us the ability to deliver software with speed, scale and agility. An investment in writing and maintaining quality test code and setting up a server to run them, together with building a test-automation mind-set and discipline within your team, fast tracks the delivery of successful software solution that becomes a true asset – able to be enhanced and modified by different people with confidence well into the future.
Here’s some of the key ways that we believe test automation delivers value:
- Faster feedback: knowing immediately whether a change to the code base has introduced a new bug.
- Ship features early, and often: enabling Continuous Delivery to reduce development and test cycle times.
- Known cost of change: making confident estimates of the effort to deliver a change or new feature aids planning and prioritization to keep the project on track.
- Frees QA people to do their job: trying to break the application to deliver a high quality solution.
- Promotes simple design: writing code with testability in mind encourages simplicity, loose coupling and maintainability.
- Enables Behaviour-driven Development (BDD): writing tests that are specifications of behavior, as part of a whole team focus on BDD, emphasises real business value and improves team communication and collaboration.
Test Automation Done Well
We’re not in favour of buying a commercial test–automation tool, instead we recommend using one of the mature open source tools and APIs available which offer great value and provide lots of community support.
Important qualities of a well-functioning test-automation approach are:
- Tests are written at different levels. There are unit-tests, integration-tests and acceptance tests. All offer different levels of granularity, and together they should provide ‘enough’ of a safety net.
- Open source test tools are being utilised. Using mature offerings like rspec, xUnit, cucumber, spock & geb, means that you’re needing to write less test code to get the required coverage.
- The tests are fast to run. To get timely feedback, this is key. In particular, the more expensive acceptance tests – which simulate the end-to-end user experience– need to be written with the knowledge that it takes time to start a browser simulator, or load data onto an inbound test queue.
- Tests are being run all the time. It’s one thing for a developer to remember to run all tests before pushing changes, but having an automated process that is always detecting and merging changes and then running the full test suite is crucial. A Continuous Integration (CI) server is a must.
- Builds are being pushed into production often. CI is a great start but it’s not enough. The idea of also automating the process of promoting a build candidate through the appropriate test environments (eg. dev, sit, performance, uat) -with automated testing baked in – is the only way to be able to push features live as often as required. This also reduces the waste associated with managing‘inflight’ releases allowing the team to focus on delivering the next thing the client needs.
- The tests are not trying to cover everything. It’s a mistake to think that by trying to cover every corner case, the team will be able to go faster. There is a cost in maintaining the suite of tests and there is a tipping point we’ve learnt through experience. All main-use and high-value flows are covered, as are areas of high business logic complexity, but then it becomes a risk-assessment judgement call.
- Tests are covering more than the functional requirements. Software solutions have different profiles based on the key ‘non-functional’ requirements. An assessment of the level of test-automation appropriate to validate throughput and performance, security, availability aspects needs to be performed.
- Test code as a first class citizen. Simply, it needs to be just as well-structured, decomposed and maintainable is as the production code it tests, in order for the team to keep going fast.