According to Wikipedia, Data-Driven Testing(DDT) is a software testing methodology that is used in the testing of computer software to describe testing done using a table of conditions directly as test inputs and verifiable outputs as well as the process where test environment settings and control are not hard-coded.
In the simplest form, the tester supplies the inputs from a row in the table and expects the outputs which occur in the same row. The table typically contains values which correspond to boundary or partition input spaces.”
If done manually, the process of data-driven testing is really time-consuming. It is usually used for regression testing or with test cases where the same test scripts are run by providing huge test data as input, sequentially.
Therefore, the requirement of automating the process of data-driven testing arose and today we have several automation-testing tools and automation testing frameworks for this purpose.
What is data-driven automation testing?
In data-driven automation testing, the test-script/ test case code is kept separate from the test data set. This separation is usually made by storing them at different locations and by having no dependency on each other. Test scripts and test data both are independent of each other and can be stored, modified, added and deleted without any implications on each other.
The order of actions in a DDT (Data-Driven testing) process is as below–
1. Read the input data from the data source, using parameterised variables in the automation test script.
2. Fill in the input test data in the AUT(application under test).
3. Perform the action according to the test script
4. Compare the expected and actual output
5. Mark the test case pass or fail
6. Continue the above steps with the next input test data from the data source.
The expected output is also entered in the test data sheet or file. So that the automation script reads the expected output, compares it with the actual output and accordingly marks the test case – Pass or Fail.
Test data storage can be achieved by any of the below options-
1. Excel files
2. CSV files
3. XML files
4. Database tables
5. Arrays
6. Text files
Tools/ frameworks for data-driven automation testing-
1. Selenium, TestNG, Apache POI (for reading and writing in Excel sheet)
2. QTP
3. Testsigma
4. Katalon Studio
5. TestComplete
Advantages of data-driven automation testing
Let us discuss the top benefits of using a data-driven automation testing framework or any DDT tool.
1. Reusability
Once we have test scripts ready, the same test script can take any number of test input, present in the files as test data. There is no need to make any changes to the script since it will just read the test data from the file and execute the test case. Whereas, if the test data is hardcoded in the test scripts, the lines of code increase tremendously and such coding practices compromise on the reusability of the program.
2. Enhanced clarity
Since there are separate folders/locations of the test data and test scripts, there is clarity in the development and maintenance of both. If test data needs any changes then it can be done easily without referring to the testing scripts and vice-versa.
Image shows separate Test Data and Test Scripts.
3. Maintainability
If there is any update required either in test scripts or test data, it is hassle-free because both test data and test scripts are placed separately with no dependency on each other. While the automation test engineer is working on the test scripts, any team member with knowledge of AUT can maintain and update the test data file without any dependency on the test scripts.
4. Broader test-coverage
With high-quality test data which perfectly reflects the real production data, we can achieve a very good test-coverage. This helps in achieving high quality and better stability of the application. The result will be a very few defects in the production environment because all the possible data is already tested and issues have been fixed accordingly.
5. Time-efficient
The automated test-run can be set up for a nightly run when there is no one working. The results can be verified in the morning, and hence a great amount of time can be saved in the testing activities. Also, if compared with manual testing, data-driven automation testing is faster because all the benefits of automation testing are underlying.
6. Reduced manual error
Unlike, manual testing the test-data is read from the files and not manually entered in the application. Human error is unavoidable when a large amount of data is being entered. Data-driven automation testing reduces the chance of manual error to a great extent and a high-quality product is a result.
7. Reduced test scripts
Because we are not hard coding the test data in the script logic, the size of the code written is reduced considerably. The same test script can be used for a huge set of test data.
8. Better use of human skills
The redundant work of manually entering the test data is monotonous and time-consuming. Human skills can be used for better purposes like exploratory testing when data-driven automation testing is in place.
9. Faster decision making
Due to the faster speed of automation and quick execution of a broader data set, the management and defect-related decisions could be concluded faster. In the Agile and DevOps environment, shorter development cycles need quick decisions and quick resolution of issues. The data-driven automation testing aids the Agile and DevOps intricacies to a great extent.
10. Independent of application development
The data-driven automation testing framework can be developed in parallel or before the actual application development. Since, we will be dealing with parameterised variables, independent test data sheets or files and separate test scripts.
The framework development is independent of the actual AUT. If an automation tool like testsigma is being used for the DDT, this process becomes easier. To create test scripts for reading/writing test data from files and scripts of test functions need expertise and effort.
11. Minimum technical knowledge required for test data creation
We have separated the test scripts and the test data in the data-driven automation testing process. The scripting part requires expertise and technical knowledge; however, the test data creation does not require any technical expertise. A person who has knowledge of the AUT and domain would be able to fill in the test data values in the sheet/file.
Conclusion
As the name suggests, data-driven testing (DDT) means testing which is driven by data. Data is of the extreme importance here in data-driven automation testing. A judicious selection of the test data thus becomes imperative in this case.
Simply running the automation scripts on a massive test data set, without a proper vision is such a waste of time, effort and resources. A proper understanding of the AUT and a very good domain knowledge prepares the background for a great test data set.
In current times data is everything- Big Data, AI (Artificial Intelligence) and ML (Machine learning) endorse this fact with the highest splendour.
Testing can not be any different, to cover a broader spectrum of scenarios in the software testing – a diligent selection of test data is needed. So that any possible defect is already taken care of during testing and the risk of any potential defect in production is minimised.
Improve test coverage with positive and negative test data sets using Testsigma
