Agile Testing - Tracking Activities

Test Status can be communicated −

• During daily stand-up meetings
• Using standard test management tools
• Via messengers

Test status determined by test passing status is crucial in deciding the whether the task is “Done”. Done means all the tests for the task pass.

Test Progress

Test Progress can be tracked using −

• Scrum Boards (Agile Task Boards)
• Burndown Charts
• Automated Test Results

Test Progress also has a direct impact on development progress. This is because a User Story can be moved to Done status only after the Acceptance Criteria is reached. This, in turn, is decided by Test Status as the Acceptance Criteria is judged by a Test Status.

If there are any delays or blockages in test progress, the entire team discusses and works collaboratively to resolve the same.

In Agile Projects, changes take place quite often. When many changes take place, we can expect that the Test Status, Test Progress and Product Quality to evolve constantly. The Agile testers need to get that information to the team so that the appropriate decisions can be made at the right time to stay on track for successful completion of each iteration.

When changes happen, they can affect existing features from previous iterations. In such cases, manual and automated tests must be updated to deal effectively with regression risk. Regression testing is also needed.

Product Quality

Product Quality Metrics include −

• Tests Pass / Fail
• Defects Found / Fixed
• Test Coverage
• Test Pass/Fail Rates
• Defect Discovery Rates
• Defect Density

Automating the gathering and reporting of product quality metrics helps in −

• Maintaining transparency.
• Gathering all the relevant and required metrics at the right time.
• Immediate reporting without communication delays.
• Allowing testers to focus on testing.
• Filtering misuse of metrics.

To secure overall product quality, the Agile team needs to obtain customer feedback on whether the product meets customer expectations. This needs to be carried out at the end of each iteration, and the feedback will be an input for subsequent iterations.

Key Success Factors

In Agile projects, quality products can be delivered if Agile testing is successful.

The following points need to be considered for the success of Agile testing −

• Agile testing is based on test first and continuous testing approaches. Hence, the traditional testing tools, which are built on test-last approach, may not be suitable. Hence, while choosing the Testing Tools in Agile projects, the alignment to Agile testing needs to be verified.

• Reduce total testing time by automating tests earlier in the development lifecycle.

• Agile testers need to maintain their pace to align to the development release schedule. Hence, proper planning, tracking, and re-planning of the testing activities need to be done on the fly with product quality as the goal.

• Manual testing accounts to 80% of the testing in the projects. Hence, testers with expertise are need to be part of the Agile team.

• Participation of these testers with expertise throughout the development lifecycle makes the entire team focus on quality product meeting customer expectations.

  • Defining user stories emphasizing product behavior expected by the end users.

  • Identifying Acceptance Criteria at user story level / task level as per customer expectations.

  • Effort and duration estimation for testing activities.

  • Planning testing activities.

  • Aligning with the development team to ensure production of code that meets the requirements with an upfront test design.

  • Test first and continuous testing to ensure that done status is reached meeting the acceptance criteria at the expected time.

  • Ensuring testing at all levels within the sprint.

  • Regression testing at the end of each sprint.

  • Collecting and analyzing product metrics that are useful for the success of the project.

  • Analyzing defects to identify which need to be fixed in the current Sprint and which can be delayed to subsequent Sprints.

  • Focusing on what is important from the Customer’s point of view.

Lisa Crispin has defined seven key Factors for Agile Testing Success −

• Whole Team approach − In this kind of approach, the developers train the testers and the testers train other team members. This helps everyone to understand every task in the project, thereby collaboration and contribution will have maximum benefit. Collaboration of testers with customers is also an important factor to set their expectations right at the beginning and translating the acceptance criteria to the required to pass the test.

• Agile Testing Mindset − The testers are proactive in continually improving the quality and collaborating constantly with the rest of the team.

• Automate Regression Testing − Design for testability and drive development with tests. Start simple and allow the team to choose the tools. Be ready to provide advice.

• Provide and Obtain Feedback − As this is a core Agile value, the entire team should be open for feedback. As the testers are expert feedback providers, need to focus on relevant and necessary information. In return, on obtaining feedback should accommodate test case changes and testing.

• Build a Foundation of Core Agile Practices − Focus on testing alongside coding, continuous integration, collaborative test environments, working incrementally, acceptance for changes, maintaining synergy.

• Collaborate with Customers − Elicit examples, understanding, and checking the requirements mapping to the product behavior, setting up Acceptance Criteria, obtaining feedback.

• Look at the Big Picture − Drive development with business-facing tests and examples using real world test data and thinking about impacts on other areas.