Selenium was first conceptualized in 2004 at ThoughtWorks for website automation and has come a long way from it’s first Web Driver version to becoming a W3C standard protocol. With a single interface that lets you write test scripts in programming languages like Ruby, Java, NodeJS, PHP, among others and being open-source, Selenium has gained quite a lot of popularity so much so that it stands tall among the most popular commercial tools today.
But for anyone who has worked on Selenium, it is not all easy. If the number of issues open at the moment and the number of questions being posted on forums like StackOverflow is any indication, the number of problems being faced by software testers using Selenium does not seem to be decreasing.
The test cases are brittle, the failures are common and the maintenance of these automated test cases takes more time and cost than it takes to automate them. However, with a very strong community and integrations to make it better and workable, this can be done. But if you are working in tight timelines, this wouldn’t be the ideal way to go about.
If we start thinking about why Selenium tests fail, an obvious reason would be that the websites and the technologies used to build them have been evolving at a fast pace. And the number of additions to the features and functionalities has only been increasing exponentially.
Keeping up with the changes and making sure the tests run reliably every single time is not an easy job.
Let us look at a few reasons why Selenium tests fail:
- Asynchronous websites
- Bugs
- Dynamic websites that change without refresh
- Small changes in the website source code
- Timeouts
- Alerts, Pop-ups, Nested IFrames
- Selenium WebDriver version mismatch
- ElementNotFoundException Sometimes during automation, testers miss out on accounting of small components like the page load progress icon and when the script executes, it fails to find the element that will load after the page loads. These situations can easily be solved by accounting for the page load and waiting for the page to load completely. Sometimes an implicit wait for the required element to load can also solve such problems.
- TimeOutException
This exception appears when Selenium waited for some operation to complete or for an element to appear or disappear but it did not happen during the time it was expected to wait. The time it is supposed to wait can be configured as well.
- Changing location for Selectors
- Difficult to set up initially
- Substantial time is taken to automate
- Substantial time is taken to maintain the test cases
- Pass/Fail Reports Selenium does not offer rich reports on its own and requires third party integrations for more than just Pass/Fail error reports.
- Testsigma
- TOSCA
- Katalon
- TestComplete
- Dedicated technical support
- Easier Test Case Creation
- Automatic Test Case maintenance
- Easier to set up
- Low Learning Curve
Selenium tests are developed keeping a particular order in mind and when there are websites that are built using asynchronous architecture. The order of response cannot be fixed at all times. In such scenarios, sometimes the responses from the website are not in order. As a result, the test case fails. Such failures are tough to debug as well because they are not reproducible at all times.
With websites rapidly evolving, multiple languages and libraries are available for website development. There are a lot of features that needs to be supported.
Selenium does come out with a better version frequently supporting any new change that comes up but they also leave out bugs and sometimes these bugs appear while people are trying to automate their websites.
Now websites are built that do a page reload or refresh rarely and, all the changes happen dynamically using JavaScript on the same page. In some scenarios, if there is a small error in one step, the whole test case fails because there is no opportunity to reload or refresh the page.
The most common example of such a change is when a unique ui identifier(also known as locator) of an element on the web page changes. When such a change is checked in, all the test cases that were referring to this unique ui identifier fails and needs to be fixed. Sometimes small changes are passed on to the code that make the test cases fail but Selenium does not report the exact reason for failure and it makes the test cases seem flaky.
Timeout errors where Selenium fails while waiting for a component to load are very common while automating using Selenium and can be handled by adding appropriate “waits”. Just ensure that explicit waits are only added where necessary because that increases the overall run time of the test case. Implicit waits are better to use in such scenarios.
Some common errors that most Selenium users may have come across but are easy to fix:
Selenium Users faced problems while automating alerts, pop-ups and Nested IFrames related functionalities on a website but with the latest version of Selenium, this problem has been handled quite well. If there is anything that cannot be handled, it is the OS level alerts because Selenium can only handle website related operations.
Selenium releases WebDrivers for supported browsers to execute tests on them. If the installed browser on the system does not match the versions supported by the web drivers, an exception like this is reported: “This version of ChromeDriver only supports Chrome version 74”. The solution is to either update the Selenium WebDriver to support the installed browser or update the installed browser to the version supported by selenium webdriver and the error will go away.
Sometimes when a site is responsive, the location of the selectors on a web page changes with change in size. This results in test cases failing with the web page size. Such failures can be fixed by using a selector that does not change with changing size for eg. using unique names or ui identifiers as selectors.
Above are the most common exceptions that people face but if you are looking for a more detailed list, check here. These exceptions may occur while testing using any automated testing tool but there are certain limitations of Selenium that are detailed below.
Limitations of using Selenium for Automation:
Selenium works with certain programming languages. Elaborate setup for development in the programming language and support for Selenium needs to be prepared before automation can be started.
Because every step has to be coded it takes a good amount of time for just automating a simple workflow.
If there are failures in test scripts, it may take a long time to debug and figure out the root cause of failure which also means more cost.
Alternatives to Selenium
Instead of spending a lot of time, money and effort on getting things to work with Selenium, try using other tools that are there in the market and have dedicated technical support, some of such tools are:
Testsigma is an NLP based automated testing tool that lets you create automated tests easily and efficiently in just plain English.
This AI-powered automated testing tool is an ideal alternative for Selenium with all the flexibility that enterprise teams need.
With automatic object recognition and test case maintenance, TOSCA is a popular automated testing tool. It maximises reusability and increases maintainability with model-based test automation.
With a Selenium-based engine, Katalon is designed to create and reuse automated test scripts for UI without coding. It is free to use. It also has the option to automate complex scenarios using JAVA or Groovy.
Tests can be recorded, scripted or manually created with keyword driven operations and used for automated playback and error logging.While Selenium can only execute tests in a browser (and mobile apps using Appium), TestComplete supports Windows applications.
With the support of custom extensions and plugins, TestComplete functions can be extended as per need.
Benefits of using above tools:
If you face a problem you will always have support available who will give you a timeline to solve any issue you face. They will also help you figure out the issue in case are stuck. Thus you won’t be left alone in the wild looking for solutions when you need immediate help.
These automated testing tools do not demand scripting/programming skills. Katalon and TestComplete use a record and playback approach for test case creation while TOSCA automatically recognises the objects in the application and then test cases are created using the Model based approach. Testsigma on the other hand lets you write automated tests just using plain English!
Testsigma, TOSCA, TestComplete have an AI-powered core that enables automatic maintenance of test cases whenever some changes in the website cause the test cases to fail.
Testsigma is a cloud-based automated testing tool that hosts all automated tests on the cloud so you don’t need to spend any time or money on initial setup. There is also support for local executions(Hybrid) on your local devices. In comparison, Testsigma is quite easy to get started.
The special focus of all the above testing tools is the ease to learn them. Thus a lot of time and money that is spent on learning new tools can be saved if any of these automated testing tools are used. It would be quite easy for beginners into automation to get accustomed with these automated testing tools.
Selenium is still a mainstream automated testing tool since its inception but now the times are changing and when there are new technologies like AI and ML being employed, teams should explore them to actually know what they have to offer.
Since development workflows are now more complex than they have ever been, and will probably grow yet more complex over time. Selenium is poised to remain relevant for modern application testing. Depending on the requirements, we should be more open to exploring more automated testing tools and choose the relevant one that properly fit our testing requirements.
Business vector created by freepik – www.freepik.com
