In today’s world where ‘data is the new oil’ ( as said by Clive Humby), not giving proper attention to data-driven testing is not justified. If you have an application that needs data input in some form then it will require data-driven testing.
Additionally, when during testing your application – there are testing scenarios that require multiple permutations/combinations of input data in your application, then you might have to think of using a test automation tool for automating them.
We seek help from automation tools for data-driven testing, because manually it may be tiring, time-consuming, outright boring, and prone to human errors. So we create a data repository where all of the data resides, separate from the test scripts.
The data repository could be a CSV file, excel sheet, DB table, etc. Scripts and data are kept separate from each other to increase the maintainability, simplicity, and understandability for future use. The test results are a huge set of data and they need to be matched against the expected results, which are again stored in files.
Let us see a few scenarios where data-driven testing is useful in providing a quality product.
Scenario 1: Tabular data
Example: Registration forms that require email, password, and other information.
It is a classic scenario where we require data-driven testing(DDT) to perform thorough testing on the input data. The registration form needs to be tested for all sorts of inputs such as – alphanumeric, alphabets, special characters, blank input, numerics, etc.
DDT needs to be performed for negative and positive test cases as depicted in the table below:
Username valuePassword value
ValidValid
ValidInvalid
InvalidValid
InvalidInvalid
ValidNULL
NULLValid
NULLNULL
The steps will be:
- Create and store the test data in a data repository. The data file will also contain the expected output and actual output.
- The driver script will pick the input data from the files and fill in the input fields in real-time.
- After performing the logic execution the driver scripts pick the actual output from the application and write the actual output in the data file.
- Later the expected output and actual outputs are compared and the test results are recorded accordingly as Pass/Fail.
- Repeat with the next test data.
A detailed DDT guide having useful and vast information is here.
Scenario 2: Data Arrays
Example: Travel itinerary/ Flight booking systems
For testing a travel/flight booking system, we would need many permutations and combinations of the test data. The bookings will vary according to the input data such as first name, last name, origin, destination, date of travel, etc.
Interestingly, in this example, the data is difficult to store in tabular form and will be best stored in arrays called ‘flight’ and ‘passengers’. There are so many input fields in this scenario, and we need to create data sets for each of the input fields accordingly.
The driver script will read the data from the ‘flight’ and ‘passenger’ arrays and the logic of flight booking will be executed. Finally, the test results are generated based on the comparison of actual results and expected results.
Scenarios 1 and 2 are the ones used for end-to-end test automation mostly, and it is advisable to use an end-to-end test automation tool that supports data-driven testing.
An example of such a tool is Testsigma, a cloud-based tool that simplifies your test case creation along with your data-driven testing. Not only does Testsigma really eases the setup of your data-driven testing, being cloud-based gives you the ability to start with your automation soon after you signup. It’s not just that, Testsigma readily integrates with your CI/CD workflow.
Opt for quick and efficient data-driven testing with Testsigma
Read about the simplicity of data-driven testing with Testsigma here.
Scenario 3: Load and performance testing
Example: E-commerce applications
During festive sales, the e-commerce applications get record hits by the users. The application/ website should be ready for this large user interaction, we achieve this by load and performance testing.
Here, we need to create thousands of simulated user profiles, which will add the items/goods in their shopping cart and perform the purchase. For this purpose, we seek help from DDT, we will create such a huge amount of user data and user actions in the data repository.
Since it is a huge dataset we will take the help of automation to create this test data. The driver script and load/performance testing automation tools will work in tandem to run these test cases.
Scenario 4: Image-based data-driven testing
Example: Image classification algorithms
Photo management applications, e-commerce websites, search engines, healthcare, disaster monitoring, robotics, etc. need image classification algorithms to recognize and search images.
These algorithms are capable of learning about the image and eventually recognizing it. Based on this recognition the software executes the logic, for a great user experience.
The functionality of these algorithms is tested using the image-based DDT. A large dataset of images is used to verify that the algorithm can learn and recognize images correctly.
Read more about how to select test data for these algorithms here.
Scenario 5: Data-driven Testing for testing NLP(Natural Language Processing)
Example: Chatbots
In quest of providing an incredible user experience, organizations are creating their software to support NLP(Natural Language Processing). Most of the websites are providing chatbots to their customers for resolving their queries in Natural Languages.
How do chatbots gain such a high level of consistency, responsiveness, correctness?
Through qualitative research data is gathered using surveys, questionnaires, user observations, clickstreams, etc. Then this data is fed into the chatbot for machine learning purposes, and then we use this test data for data-driven testing.
The data provider will provide the user queries into the chatbot and then we can verify the correctness of the response during testing. Also, we would like to test the response time of the chatbot using this DDT.
Scenario 6: Spam detection algorithm
Example: email spam
No one wants spam emails, so spam detection algorithms have been designed to identify such spam emails. For this purpose the data is collected, analyzed, and a data set is created for the algorithm.
A part of this data will act as the training data for the spam detection algorithm and the other part will be used as test data.
The test data is then run on the algorithm to decide the accuracy, predictability, and precision of the spam detection algorithm. Here also we are using data-driven testing to test the accuracy of the email spam detection algorithm in the application.
Conclusion
Machine learning and Big Data are driving the major industry decisions today. We all know very well that in today’s scenario, data is the new oil. To test such a huge amount of data we need help from DDT and tools that support it.
Data-driven testing provides enormous benefits, and in many cases, it is the only option to perform testing. The benefits of DDT are reusability, simplicity, maintainability, better test coverage, etc. Read more about the benefits of DDT here.