Nowadays, it is difficult to imagine the Quality Assurance (QA) process without automation tools - their number on the market is constantly growing, and we are continually discovering new options. However, some QA engineers still prefer the "manual" path. Why?
There can be several reasons—the desire to stay in your comfort zone, fear of difficulties with implementing and mastering new tools, or reluctance to test new solutions when the old methods seem to work well.
Nevertheless, it is worth remembering that the implementation of automation allows for better optimization of processes, reduces costs, and saves time for the development of QA engineer skills and qualifications. Below you will find a list of tools that will improve (and facilitate) your work and meet modern project challenges.
If you work with the web, API, or GUI daily, and at the same time you do not have much programming experience, some of the following solutions may be beneficial:
PyTest is another open source tool that allows you to run tests (both unit and integration, or end-to-end) written in Python. It is valued as a web service testing tool.
As an advantage, it is worth mentioning the universality of this tool, regardless of the complexity of the tests—PyTest is perfect for writing both simple tests and more advanced ones.
PyTest supports "fixtures"—functions used to set preconditions and postconditions. It is worth knowing that fixtures have the scope to allow for a single execution of a common condition for a defined group of tests—this can help save time (e.g. when creating databases).
PyTest also allows for test parametrization, which is useful when the same test is used with different inputs.
The advantages of PyTest include the availability of the documentation and the multitude of libraries, the possibility of extending with plugins (pytest-xdist allows for parallel testing), and a large and active community helping to improve this tool.
The popularity of PyTest is also visible in the possibility of integrating this tool with other popular solutions, such as PyCharm—tests can be efficiently run directly in the IDE and checked in the IDE debugger, thus finding bugs quicker.
PyTest is not a friendly solution for employees with little programming knowledge—in this case, knowledge of Python is an absolute "must-have." PyTest requires excellent coding skills and a commitment to learning it, making it a less accessible tool for some coworkers.
Cypress (Cypress.io)—this is a tool that facilitates the automation of integration tests and the user interface. Cypress Dashboard Service complements Test Runner, making it easier to view current tests. It is handy during tests in the CI pipeline—it helps with pipeline optimization and, what's more, it allows you to run several tests in parallel.
Cypress allows recording and adding screenshots (Cypress snapshot module) of the tests performed. Some of the most significant advantages of this solution is its speed, ease of debugging, and code writing. It is supported by various browsers—Chrome, Firefox, Edge, and more. And last but not least—it does not require complicated configuration.
Cypress.io offers intercepts, stubs, executing CLI commands, compatibility with mochawesome reporters—in short, everything you need for effective testing.
What's more, this tool has clear and comprehensive documentation, and an active and growing community that supports the constantly increasing number of packages for the plug-in system and extensions.
This is one of the most popular and widely used tools on the market. Selenium has a developed and active community that will help find answers to many problems, or we can use the proven solutions available on forums. In addition, Selenium can be integrated with, among others, Jenkins or TravisCI. The many available libraries and plugins will undoubtedly help in your everyday work.
Selenium offers a built-in step recording mechanism that allows you to create new test cases without programming skills (however, having such skills helps to use the tool's capabilities fully). Unfortunately, each change on the browser side necessitates recreating the same test cases.
One of the most significant advantages of this tool is the ability to transfer test data from code to an external file, e.g. CSV, which improves its maintainability. What's more, Selenium allows you to run tests in headless mode, which can increase performance (in the case of Selenium, each browser loading is extra seconds that we lose just by showing the interface) or reducing resource consumption.
Cucumber is mainly used to create real-time acceptance tests. It is based on the Behavior Driven Development (BDD) methodology, which makes it more accessible and easy to use for non-technical team workers.
The tool uses the specially created Gherkin language in which the application functions are written.
Cucumber is an open source framework, appreciated for its compatibility with many programming languages and ease of integration with other tools, e.g. Selenium.
The biggest complaints about Cucumber are that it adds unnecessary complications. Testing a new feature requires checking if the already existing test steps can be used.
Working with Cucumber and Gherkin is a balance between creating enough generic functions to be reusable and, at the same time, being specific enough to differentiate between various test scenarios.
Katalon - a tool supporting API, web, and mobile tests. It does not require programming skills, but the language used to write tests is Groovy, which can be a barrier to getting started with this tool.
Katalon, like Selenium, can be easily integrated with external tools such as Jenkins, Git, or JIRA and allows you to record tests or take test data to an external source. What distinguishes Katalon from Selenium? Katalon enables API testing, which is a frequent source of communication during testing. The reports provided by Katalon are user-friendly and contain helpful information. It is also worth mentioning the more accessible and easier-to-learn GUI.
Robot Framework is an open source Python-based solution that uses Keyword Driven Testing (KDT). In KDT, test cases are described by predefined keywords. Keywords are names related to a set of actions needed to execute a specific test case step. KDT can be executed without programming knowledge, which makes it easier for the non-technical members of the team. It helps to automate integration, GUI, API, or component tests.
The Robot is appreciated for its speed, clear reports (they have only one template), a wide range of libraries, the ability to integrate with CI/CD pipeline tools (e.g. Jenkins), and the option for an engineer to tag tests/suites. Ease of use is often marked also for non-technical individuals at the primary level (thanks to KDT). Still, a lack of programming knowledge may result in the creation of a suite that is difficult to maintain.
Robot Framework works best as a test runner (for more complicated tests), when the test is based on Python. This tool does not offer built-in debugging capabilities, and there is also limited support for Intellisense and IDE.
Although Robot Framework has most of the functions of other programming languages, including the more complex ones (conditional instructions, loops, operations on various data types and objects), using them is cumbersome and the code created is hardly readable. Because of that, it is challenging to apply good programming practices (e.g. it promotes global variables).
Due to the lack of built-in tools and limited support for Intellisense and IDE, debugging is also complicated. To avoid these problems and keep the test definitions readable, it is advisable to put all the "logic" in Python, define high-level keywords, simplify, and increase the code's readability at the Robot level.
Sikuli - a tool for automating GUI and identifying GUI elements using the image recognition method. Sikuli is especially useful for automating repetitive actions; e.g. simulating keyboard actions or searching for elements in an image. Tests are automated with pictures, which speeds up the process of creating tests - however, you should remember that changing the picture requires changing the entire script.
Just by clicking the tool, simple and repetitive operations can be automated without programming. Everything is due to a transparent IDE that facilitates the creation of scripts. You can see how simple this tool is by watching sample tutorials. More complicated scripts require knowledge of Python, but learning the basics of the language should not be difficult.
How does Sikuli compare to Selenium? A functionality that Sikuli offers, which cannot be found in Selenium, is automating Flash objects. This tool also allows you to test the web GUI and desktop web applications. The good news is that it is possible to integrate these two tools and use their potential simultaneously.
STAF/STAX - an open source, cross-platform framework for executing test scenarios on several machines which allows you to write your scripts.
Test scenarios are written in XML, which can often be challenging to read and easy to make mistakes, e.g. by leaving a tag open. STAF/STAX uses services such as LOG, which provides a set of operations related to logging, e.g. writing to a file or searching in a file.
The main advantages of this tool are its maturity as a product and its support for many programming languages. This framework will work whether you use Java or Python.
The above list is just a few of the extensive list of tools available - we decided to cover the ones we consider worth your attention and most helpful in your daily work. The choice is based on our personal experience - we use these solutions ourselves in QA automation projects in CodiLime. We hope that this article has addressed any doubts and you are already installing the selected tools, adding something new and fresh to your toolkit.