STLC - Test Execution
Test execution is the process of executing the code and comparing the expected and actual results. Following factors need to be considered for a test execution process −
- Based on a risk, select a subset of test suite to be executed for this cycle.
- Assign the test cases in each test suite to testers for execution.
- Execute tests, report bugs, and capture test status continuously.
- Resolve blocking issues as they arise.
- Report status, adjust assignments, and reconsider plans and priorities daily.
- Report test cycle findings and status.
The following points need to be considered for Test Execution.
In this phase, the QA team performs actual validation of AUT based on prepared test cases and compares the stepwise result with the expected result.
The entry criteria of this phase is completion of the Test Plan and the Test Cases Development phase, the test data should also be ready.
The validation of Test Environment setup is always recommended through smoke testing before officially entering the test execution.
The exit criteria requires the successful validation of all Test Cases; Defects should be closed or deferred; test case execution and defect summary report should be ready.
Activities for Test Execution
The objective of this phase is real time validation of AUT before moving on to production/release. To sign off from this phase, the QA team performs different types of testing to ensure the quality of product. Along with this defect reporting and retesting is also crucial activity in this phase. Following are the important activities of this phase −
System Integration Testing
The real validation of product / AUT starts here. System Integration Testing (SIT) is a black box testing technique that evaluates the system's compliance against specified requirements/ test cases prepared.
System Integration Testing is usually performed on subset of system. The SIT can be performed with minimum usage of testing tools, verified for the interactions exchanged and the behavior of each data field within individual layer is also investigated. After the integration, there are three main states of data flow −
- Data state within the integration layer
- Data state within the database layer
- Data state within the Application layer
Note − In SIT testing, the QA team tries to find as many defects as possible to ensure the quality. The main objective here is finding bugs as many as possible.
Defect Reporting
A software bug arises when the expected result doesn't match with the actual result. It can be an error, flaw, failure, or fault in a computer program. Most bugs arise from mistakes and errors made by developers or architects.
While performing SIT testing, the QA team finds these types of defects and these need to be reported to the concerned team members. The members take further action and fix the defects. Another advantage of reporting is it eases the tracking of the status of defects. There are many popular tools like ALM, QC, JIRA, Version One, Bugzilla that support defect reporting and tracking.
Defect Reporting is a process of finding defects in the application under test or product by testing or recording feedback from customers and making new versions of the product that fix the defects based on the client’s feedback.
Defect tracking is also an important process in software engineering as complex and business critical systems have hundreds of defects. One of the most challenging factors is managing, evaluating and prioritizing these defects. The number of defects gets multiplied over a period of time and to effectively manage them, defect tracking system is used to make the job easier.
Defect Mapping
Once defect is reported and logged, it should be mapped with the concerned failed/blocked test cases and corresponding requirements in Requirement Traceability Matrix. This mapping is done by the Defect Reporter. It helps to make a proper defect report and analyze the impishness in product. Once the test cases and requirements are mapped with the defect, stakeholders can analyze and take a decision on whether to fix/defer the defect based on priority and severity.
Re-testing
Re-testing is executing a previously failed test against AUT to check whether the problem is resolved. After a defect has been fixed, re-testing is performed to check the scenario under the same environmental conditions.
During re-testing, testers look for granular details at the changed area of functionality, whereas regression testing covers all the main functions to ensure that no functionalities are broken due to this change.
Regression Testing
Once all defects are in closed, deferred or rejected status and none of the test cases are in progress/failed/no run status, it can be said that system integration testing is completely based on test cases and requirement. However, one round of quick testing is required to ensure that none of the functionality is broken due to code changes/ defect fixes.
Regression testing is a black box testing technique that consists of re-executing those tests that have had an impact due to code changes. These tests should be executed as often as possible throughout the software development life cycle.
Types of Regression Tests
Final Regression Tests − A "final regression testing" is performed to validate the build that has not undergone a change for a period of time. This build is deployed or shipped to customers.
Regression Tests − A normal regression testing is performed to verify if the build has NOT broken any other parts of the application by the recent code changes for defect fixing or for enhancement.
Activity Block Diagram
Following block diagram shows the important activities performed in this phase; it also shows the dependency from the previous phases −
