The Different Software Testing Types Explained

According to IEEE Std 829-1998, Software Testing is defined as “the process of analyzing a software item to detect the differences between existing and required conditions (i.e. defects) and evaluating the features of the software items.”

With the advent of new technologies and new software being released every day, software testing is gaining more strength and accolades. Today, testing is imperative and there is no way we can skip it. Testing and development are going hand in hand for a faster release in this Agile world.

Everyone wants a stable, high-quality and functionally correct software which meets all the requirements it was intended to fulfill. And all this can be accomplished with proper Software Testing which uncovers any hidden and potential error which may be problematic later for the software and subsequently to the clients or the organization.

So, let’s first discuss testing techniques before we move to software testing types.

Testing Techniques

On a broader level, testing techniques can be divided into two types, based on the state of the application(static or dynamic).

1. Static Testing (Verification): Verification implies, “Are we building the product right?” 

In Static Testing, code is not executed. Documents like requirements documents, design documents, code are reviewed and review comments are provided at early stages of Software Lifecycle. It is a preventive measure, which eliminates defects at early stages and hence, the ROI is high.


 Examples of Static Testing-Walkthrough, Reviews, Static Code Review, Inspection.

2. Dynamic Testing (Validation): Validation implies, “Are we building the right product?” 

 In Dynamic Testing, the code is executed. The program is run and inputs are provided, the actual results are validated against expected results. Defects are found and fixed, it is performed at later stages of the Software Lifecycle. Hence, the ROI is low. If the defect is fixed, it involves more time and is more expensive.

 Examples of Dynamic Testing- Unit testing, System testing, Integration testing, Performance testing, Security testing. 

Based on the methodology or approach, testing techniques can be classified as below:

1. Black Box Testing

The Software Program is considered as a black box, and the user is not interested in the internal structure of the box. The output from the black box is validated against the input provided irrespective of the internal code structure.

Examples

• Boundary Value Analysis
• Equivalence Partitioning
• Decision Table
• Exploratory Testing
• Error guessing
2. White Box Testing

The internal structure and design of the code are known to the user; hence all the logical paths are tested under White Box testing. It is also known as Glass Box or Clear Box testing since the user knows the internal logic and structure of the code inside the box.


 Examples

• Statement Coverage
• Path Coverage 
• Decision Coverage
3. Grey Box Testing

Contains features of Black Box as well as White Box testing, where the tester has limited knowledge of the internal structure of the application e.g. data structures and algorithms. Unlike, White Box testing, testers do not need to have complete programming knowledge.

Grey Box Testing proves to be effective while performing Penetration Testing and Integration Testing. Since both, Penetration and Integration require partial knowledge of the internal structure.

Testing Types

On a broader level, testing types can be divided into two types, based on the fact that Functional testing deals with the functional specifications or business requirements and Non-Functional testing deals with the performance of the application.

However, Non-Functional testing is equally important as Functional testing is obviously, no one will want to use an application which is not scalable or secure, or let’s say can’t handle a load of many DB operations at an instance. Hence, the performance of the application in various areas is tested during Non-Functional testing.

Let’s dig deeper and find out more.

Functional Testing

1. Basic Functionality and conformance of the application with Functional Requirements are validated.
2. It is a type of Black Box testing; hence the internal structure and design of code are not checked.
3. Test cases are created, then run on the system and the actual results are validated against the expected results.
4. Any deviation from expected results is logged as defects and then fixed by the development team. 

Types of Functional Testing

1. Unit Testing- The most basic unit or component of an application is tested usually by the developer to ensure the smallest testable code is working fine. It is a White Box testing technique and appears at the bottom of the V-Model, i.e. it is the first testing type to be carried out in SDLC.


2. Integration Testing- When two or more components or units of the software are integrated together, they need to interact with each other in the form of commands, data exchange or DB calls. Integration testing is performed on them as a single cluster to check that the interaction between them is happening, as expected. 


3. Interface Testing- It comes under Integration testing, the correctness of data exchange or transfer between two components is tested in Interface testing. 

For e.g. One component creates a .csv file as output, the other connected component processes the .csv file into XML and sends to the third component. During this data transfer, the data should remain intact and all the components should be able to process the file and send the file to the next component successfully.



4. System Testing- All the modules of the application are combined and the whole system is tested as a single unit for correctness against the requirement specification. 


5. Regression Testing- Whenever there are some code fixes, or any functionality enhancement, the code is modified. Regression testing makes sure that these code changes have not injected any new defects in the code and that the previously working functionality is also intact and working.


6. Smoke Testing- Performed on initial unstable builds, whenever a new build is released by developers, testing team performs Smoke testing to be sure that all end-to-end functionalities are working. If any of the major functionality is broken due to a new build, the build is rejected and sent again to developers. This is performed to ensure that new code changes have not broken any major functionality and the build can be taken forward for the next level of testing.


7. Sanity Testing- It is normally a subset of Regression tests; it is performed whenever there is a new build released for a stable application. Only after running the Sanity test suite, the build is taken forward for the next level of testing. Difference between Smoke and Sanity is- Smoke testing is performed on an initial unstable application, whereas Sanity is performed on a stable application.


8. User Acceptance Testing- It is the last level of testing in a V-Model, parallel to the requirement analysis phase. It defines how well the application is accepted by the real end-users. The fulfilment of business requirements, by actual users, is checked, in a real environment.  
   

Non-Functional Testing

1. In Non-Functional testing, important non-functional aspects of the application are tested like Security, Scalability, Performance etc.
2. How well the application will perform under stressful conditions of Database, Users, Data Load etc.
3. From a user’s perspective, is the application usable, maintainable, scalable and secure to gain user’s confidence and trust.

Types of Non-Functional testing

1. Performance Testing- The performance of the application is measured while subjecting the application to real-world conditions. During this testing, performance parameters like response time, scalability, stability and efficiency of resource usage etc. are monitored; hence it is useful in finding out when the application will degrade. It will help in enhancing the application design or architecture in a way that reliability and fast response time is ensured.


2. Stress Testing- Application is subjected to anomalous conditions, which will not happen under normal circumstances. Load on the application is increased to the point that it breaks and the application behavior is recorded. Few questions that are answered via this testing are –  How the system behaves under stressful conditions? if it crashed will it be able to recover itself and restart?


3. Volume Testing- Aim of this testing is to ascertain how well the system behaves when a large volume of data or data operations are handled by Database. Will the database be able to store and process a huge volume of data? Will there be an issue due to the huge data volume?


4. Load Testing- The performance of the application under expected load as in the real world is measured. All possible loads on the application are simulated and the performance is checked.
   For example, on a normal day, 1000 users are expected to visit a website, the performance of the website is measured by simulating concurrent 1000 users, this is a normal load during real-world usage. However, when a promotion is running, the number of users may increase to 2000, in this case, the application is tested for a load of 2000 users also because this is expected load.
   Unlike Stress Testing, abnormal conditions are not tested, only the expected load i.e. the load that application is expected to face in the real world is tested on the system. If the application degrades under load, then changes in the architecture could be made to avoid such bottleneck.


5. Security Testing- Security of the application from the point of view of the network, data, system and the application is tested. Any non-authorized person should not be able to access the application and should not be able to access any confidential data. This testing ensures customer reliability and confidence in the application. Penetration testing is an example of Security testing.


6. Scalability Testing- In case the application needs to be scalable in future, will the architecture and design allow? For example, if we add more servers, have more database transactions, increase the user load in the future, will our application allow that? This is tested under Scalability testing of the application.


7. Usability Testing- Deals with the usability of the system from a user’s perspective. Is the user able to navigate through the application without any help? Next time, when the user visits the application, is he able to remember the application without any help or problem? How efficiently a user is able to use the system? All these questions are answered via usability testing.


8. Maintainability Testing- Designing an application is one thing, and maintaining the application for future expansion and requirements is another and equally important. How well the application accommodates the changes and updates done on the code or system, comes under maintainability testing.


9. Compatibility Testing- The compatibility of the application is tested with regard to different Operating Systems, Browsers, Hardware, Network capacity, Devices etc. The application should be able to perform well on the prioritized environments by the customer.
   

Manual and Automation Testing

If we categorise the testing services based on the effort(human or machine) we have two broad categories.

Manual Testing

As the name suggests, manual testing is carried out manually by real people. Testers perform manual testing by following the test steps in the test case; clicking, providing inputs on the application and finally validating the test results. Because humans are directly involved in this process, there could be chances of error and it will be time consuming. However, manual testing becomes imperative for scenarios where automation testing is not possible.

E.g. Usability Testing – where the tester is testing the application from the user’s perspective, or its user-friendliness.

Automation Testing

Automation testing involves the use of automation tools and scripts to run the test cases automatically, without manual intervention. These tools run the test cases, record the test results and sometimes log the defects in the defect tracking tool. Time-consuming and repetitive test cases like regression testing cases are an ideal fit for automation testing.

E.g. Selenium, Testsigma, TestComplete etc.

Conclusion

To conclude, below quote from Bill Gates defines an organization perfectly:

“We have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We’re more of a testing, a quality software organization than we’re a software organization.”- Bill Gates

And this surely emphasizes how important software testing is.

Knowledge of the various testing types and when to use which testing type is crucial for a good tester. Each has its own importance and usefulness in the testing Life Cycle of an application. A well-equipped tester with required knowledge surely wins half the battle.