Agile Testing - Scrum

Scrum advocates Whole Team Approach, in the sense that every team member has to take part in every project activity. Scrum team is self-organizing with accountability to the project deliverables. Decision-making is left to the team that results in appropriate actions being taken at the right time without any time delays. This approach also encourages proper use of the team talent instead of restricting to one activity. Testers also participate in all the project and development activities contributing their expertise in testing.

The whole team works together on Test Strategy, Test Planning, Test Specification, Test Execution, Test Evaluation, and Test Results Reporting.

Collaborative User Story Creation

Testers participate in User Story Creation. Testers contribute their ideas on possible behavior of the system. This helps in the customer and/or end user understanding the system in the real environment and thus getting clarity on what actually they want as the outcome. This results in faster freezing of requirements and also reduces the probability of changes in the requirements later on.

Testers also come up with the Acceptance Criteria for every scenario agreed by the customer.

Testers contribute to the creation of testable user stories.

Release Planning

Release Planning is done for the entire project. However, Scrum framework involves iterative decision making as more information is obtained in the due course of executing sprints. Hence, Release Planning session at the beginning of the project need not produce a detailed release plan for the entire project. It can be updated continually, as relevant information is available.

Every sprint-end need not have a release. A release can be after a group of sprints. The main criteria of a release is to deliver business value to the customer. The team decides on the sprint length with release planning as an input.

Release Planning is the basis of test approach and test plan for release. Testers estimate Test Effort and plan Testing for the release. When release plans change, testers must handle changes, obtain an adequate test basis considering larger context of release. Testers also provide testing effort that is required at the end of all the sprints.

Sprint Planning

Sprint planning is done at the beginning of each sprint. The sprint backlog is created with the user stories picked up from the product backlog for implementation in that particular sprint.

The testers should −

• Determine the testability of the user stories selected for the sprint
• Create acceptance tests
• Define test levels
• Identify test automation

Testers update the test plan with the estimates for testing effort and durations in the sprint. This ensures provision of time for required testing during the time-boxed sprints and also accountability of the testing effort.

Test Analysis

When a sprint begins, as the developers carry on story analysis for design and implementation, testers perform test analysis for the stories in the sprint backlog. Tester create the required test cases – both manual and automated tests.

Testing

All the members of the Scrum team should participate in testing.

• The developers execute the unit tests as they develop code for the user stories. Unit Tests are created in every sprint, before the code is written. The unit test cases are derived from low level design specifications.

• The testers perform functional and non-functional features of the user stories.

• The testers mentor the other members in the scrum team with their expertise in testing so that the entire team will have a collective accountability for the quality of the product.

• At the end of the sprint, customer and/or end users carry out User Acceptance Testing and provide feedback to the scrum team. This forms as an input to the next sprint.

• Test results are collected and maintained.

Automation Testing

Automation testing is given high importance in Scrum teams. The testers devote time in creating, executing, monitoring and maintaining of automated tests and results. As changes can occur any time in scrum projects, testers need to accommodate testing of changed features and also the regression testing involved. Automation testing facilitates managing of test effort associated with the changes. Automated tests at all levels facilitate achieving continuous integration. Automated tests run much faster than manual tests at no additional effort.

The manual testing focuses more on exploratory testing, product vulnerability, predicting defects.

Automation of Testing Activities

Automation of testing activities reduces the burden of repeated work and result in cost savings. Automate

• Test Data Generation
• Test Data Loading
• Build Deployment into Test Environment
• Test Environment Management
• Data Output Comparison

Regression Testing

In a sprint, testers test the code that is new / modified in that sprint. However, testers also need to ensure that the code developed and tested in the earlier sprints also is working along with the new code. Hence Regression testing is given importance in scrum. Automated regression tests are run in continuous integration.

Configuration Management

A Configuration management system that uses automated build and test frameworks is used in Scrum projects. This allows to run static analysis and unit tests repeatedly as new code is checked into the Configuration Management System. It also manages continuous integration of the new code with the system. Automated Regression Tests are run during Continuous Integration.

Manual Test Cases, Automated Tests, Test Data, Test Plans, Test Strategy and other Testing Artifacts need to be Version Controlled and require relevant Access Permissions to be ensured. This can be accomplished by maintaining the Testing Artifacts in the Configuration Management System.

Agile Testing Practices

Testers in a Scrum Team can follow the following Agile Practices −

• Pairing − Two Team Members sit together and work collaboratively. The two people can be two Testers or one Tester and one Developer.

• Incremental Test Design − Test Cases are developed as the Sprints progress incrementally and User Stories are added up.

Agile Metrics

During software development, collection and analysis of metrics help in improving the process and thereby achieving better productivity, quality deliverables and customer satisfaction. In Scrum based development, this is possible and the testers have to pay attention to the metrics that they need to.

Several metrics are suggested for Scrum development. The significant metrics are −

• Ratio of Successful Sprints − (Number of successful Sprints / Total number of Sprints) * 100. A Successful Sprint is one in which the Team could meet its commitment.

• Velocity − A team’s Velocity is based on the amount of Story Points a team earned during a sprint. Story Points are the measure of the User Stories counted during estimation.

• Focus Factor − (Velocity / Team’s Work Capacity) / 100. Focus Factor is the percentage of the team’s effort that results in finished stories.

• Estimation Accuracy − (Estimated effort / Actual effort) / 100. Estimation Accuracy is the Team’s ability in estimating the effort accurately.

• Sprint Burndown − Work (in Story Points or in hours) that is Remaining Vs. Work that needs to be Remaining ideally (as per the Estimation).

  • If it is more, then it means that the Team has taken up more Work than they can do.

  • If it is less, then it means the Team did not Estimate accurately.

• Defect Count − Number of defects in a Sprint. Defect count is the amount of defects in the software as against the backlog.

• Severity of Defects − Defects can be categorized as minor, major and critical as per their severity. Testers can define the categorization.

Sprint Retrospectives

In Sprint Retrospectives, all the team members will participate. They share −

• The things that went well
• Metrics
• The scope for improvements
• Action items to apply