During the entire SDLC phase, there are various kinds of testing performed at different stages.
Each stage of testing has different objectives to fulfill. The testing phase starts with unit testing followed by integration testing, system testing, system integration testing, acceptance testing, and regression testing.
During each phase of testing, if something does not work as per specifications mentioned, the defects that are detected by the testers are raised and assigned to the developers to fix it. Once defects are fixed, they are retested by testers to assure it works as desired. This is called Regression Testing.
But isn’t it same as Retesting? They are not. Let us discuss Regression Testing vs Retesting in detail in the article.
Regression Testing
Regression testing is a type of black-box testing. It ensures that modification or addition to the existing codebase has not adversely impacted the previously developed and tested features.
Regression testing confirms if the application is working as desired and expected after the change. Smoke and Sanity testing are often considered subtypes of regression testing.
Both functional and non-functional tests are executed while performing regression. Regression testing is never performed on a new module.
A quick explanation of the different types of testing are discussed here.
What is the Objective of Regression Testing?
The objective of regression testing is to check that the new code changes do not negatively impact the existing developed and tested functionalities of the application.
When is Regression Testing performed?
Regression testing is carried out in any of the below circumstances:
- When a change request (CR) is raised by the client, which leads to change in the codebase.
- When there is a change in the testing environment.
- When backend code is migrated to a different platform.
- Critical bug(s) is found during the testing phase and is fixed by the developer.
- A new feature is added to an existing application.
- The major concerns about performance issues and crashes are fixed by the developers.
- Patch fixes are added.
- The UI of the application was changed for better user experience.
Example of Regression Testing in Agile Framework:
The below diagram illustrates the role of regression testing in the final delivery of the product in an Agile framework:
Sprint is a set period of time during which specific work has to be completed and made ready to be shipped.
In Sprint X, Module 1 is planned to be completed.
During this phase, the developer develops Module 1. The tester performs unit testing on it, raises defect if any discrepancy occurs, retests defects.
After the issues are fixed and tested, it is deployed to a production-like environment.
In Sprint Y, Module 2 is planned to be completed and both modules are merged.
The development of module 2 is completed and both the modules are integrated. Later, different types of testing are performed to ensure developed product matches defined specifications.
- Unit testing- Module 2 is working as desired.
- Integration testing- performed to check data transfer flow between modules.
- Regression testing- performed to check the integration of both modules has not led to code break.
Finally, the product is deployed to a production-like environment.
In Sprint Z, the final product is delivered.
Development of Module 3 is done, the change requested is incorporated and all three modules are integrated.
During this phase, various types of testing are performed to guarantee the developed product works as desired.
- Unit testing- Module 3 is working as expected.
- Integration testing- performed to ensure the transfer of data between all three modules works fine.
- System testing- checks end to end flows works as expected.
- Acceptance testing- to validate the system is developed as per business requirement.
- Regression testing- performed to check the integration of all modules and modifications or enhancements in Module 1 has not led to code break.
Finally, the product is deployed to production and is ready to be used by the customers.
For more details on the Agile framework, please refer below link:
https://www.scrum.org/resources/blog/typical-sprint-play-play
What is Retesting?
Retesting is also known as confirmation testing. Retesting is to ensure the defects raised during the software cycle are fixed and work as per the desired specifications. The failed test cases are re-executed and passed.
What is the Objective of Retesting?
The objective of retesting is to check if the identified defects are fixed. Test cases linked to it are re-executed and passed.
When is Retesting Performed?
- Whenever a defect in the software is fixed, retesting needs to be carried out.
- Retesting has to be carried out prior to regression testing.
- The tester only performs retesting when the defects raised by them changes its status from ‘new’ to ‘awaiting retest’.
Example of Retesting
Below given diagram presents the process of retesting :
Step 1: Module 1 has been developed to produce output as A+B+2.
Step 2: The module is tested to check if the functionality works as expected. The output produced is A+B instead of A+B+2. The associated test case is failed and a defect is raised.
Step 3: The developer fixes the defect by making necessary changes in code. Then defect is reassigned to the tester to retest and verify.
Step 4: The defect is retested and output is matched with the expected result. The linked failed test case is re-run and passed. Finally, the defect is closed.
Testing Techniques used in Regression Testing
1. Prioritize test cases
The tester should be wise enough to trace out which area would be potentially impacted due to changes in code or additional features. While creating test suites, test cases should be tagged on the basis of priority (high, medium, low).
Based on the time frame and the business risk involved, testers can take a call on how much regression is required. Whether the end-to-end coverage would be sufficient to find any code break or we need to run the entire test suite to ensure there is no failure. Major cases in the regression package should include newly added features/functionalities and customer-facing aspects.
2. Re-execute all
This technique is usually applied when the entire code base is migrated to a different language or there is a major update in the backend system. As the name implies, in this practice you are required to re-run the entire test suite. As this activity is time-consuming and requires substantial resources, so mostly the entire test suite is automated.
3. Regression test selection
When the changes and impact are not major, it is advisable to make a set suite that covers relevant areas impacted due to change instead of re-executing the entire existing test suite. You should perform regression testing in those selected areas only.
The Testing Technique used in Retesting
There is no such technique used in retesting when defects get fixed, they are retested and corresponding cases are passed.
Why should the Regression Test Suite be Automated?
Regression testing is performed on existing functionalities that have been tested before. It increases exponentially with each passing release phase. Regression tests are usually good automation candidates, as by that time, the application becomes stable and testers are well versed with functionalities.
When the regression suite is automated, it gives testers an ample amount of time to perform ad-hoc testing in a production environment. This prevents any defect leakage on scenarios that might be missed or are not a member of the regression test suite.
The automation package should be designed in such a way that scripts are simple and easy to maintain.
There are many challenges faced while automating the regression test suite such as data challenges, frequent changes in the application, etc.
In-Sprint Test Automation Challenges for Regression Testing discusses some challenges in implementing In-Sprint Test Automation to perform Regression Testing.Testing Technique used in Retesting
There is no such technique involved in retesting, you need to verify the fix by retesting the defect and re-execute the linked failed cases to it. Automation of test cases is not required.
Test Strategy used in Regression Testing
The regression activity should be well planned and proper test strategy should be followed.
The regression test plan must include below pointers:
- Purpose of testing
- Test strategy
- Features to be tested
- Resource Requirement
- Hardware Requirement
- Software Requirement
- Test schedule
- Change request
- Entry / Exit criteria
- Entry criteria for when to start testing
- Exit criteria for when to stop testing
- Tools
Let’s talk about all of them in detail:
- Purpose: Describes how testing would be performed to accomplish the desired results.
- Test strategy: Consists of the approach, techniques that would be used while performing testing.
- Features to be tested: The features which will be members of regression are listed here.
- Resource Requirement:
- Hardware Requirements: Any hardware requirements which will be required during testing are identified, such as laptops, desktops, smartphones, etc.
- Software requirement: Which operating system, browsers will be required.
- Test schedule: Number of resources required to complete the testing activity within the estimated time.
- Change Request: What changes or new features are added to the existing application, for which regression needs to be performed.
- Entry / Exit criteria:
- Entry criteria: What all basic needs should be fulfilled to start testing such as environment, test data, etc.
- Exit criteria:It is always dependent on the risk involved. When to stop testing, how much testing is enough, all such pointers are described here.
- Tools: The section lists tools that can be used to make the job easier. As regression testing is re-execution of passed cases so mostly it is automated with the help of tools.
i. What will be completion criteria,
ii. Who will write test scripts,
iii. who will perform testing,
iv. Which tool can be used,
v. What will be the course of action, if product risk occurs like resource crunch, delay in production, etc.
Regression Testing vs Retesting Differences
As we see, Regression Testing and Retesting are not the same.
Top 5 Best Regression Test Automation Tools
Tools are used to make the job easier and simpler. Some of the widely used regression test automation tool available in the market are:
1. Testsigma
It is the finest automated regression testing tool available in the market to perform automated regression testing. It allows the user to write test scripts in normal English language which is as easy as writing manual test cases. AI identifies affected and relevant tests and suggest changes.
Key feature: Supports all platforms(web, native & desktop) and browsers. The tool provides options to prioritize test cases, build multiple test suites, schedule and run automated regression tests for each build automatically, generates comprehensive reports with a drill-down approach.
2. TestComplete
It enables you to have quick, easy creation and maintenance of regression tests across the web, mobile and desktop applications.
Key Features: Supports database, object-driven, data-driven, keyword-driven testing.
3. TimeShiftX
TimeShiftX is a time shift regression testing tool. It allows applications to travel in the past or future to help you perform temporary or date simulating testing.
Key Features: Requires no Environment Reboots & Reloads, Supports all platforms & operating systems.
4. Regression Tester
Regression Tester is a regression testing tool by Info-Pack.com. It allows you to perform remote testing of web-based applications.
Key Features: Easy creation of tests list, automatically runs test, fully-customized report generation.
5. Katalon
It is an open-source tool. It has a dual interchangeable interface for creating test cases, a manual view for the less technical users and a script view for experienced testers.
Key feature: Supports all platforms(Web, Native & Desktop) and browsers, script execution on multiple devices.
