Ensuring code quality and reliability is critical in the modern software development world. One of the possible solutions to achieving this goal is unit testing. Unit testing involves breaking down software into smaller components or units and subjecting them to rigorous testing. It helps identify bugs and errors early on and provides benefits that significantly enhance the development process.
In this article, we delve into the importance of unit testing and explore its various benefits to developers and businesses.
V-model in software development
Software development can be done in many different ways but there are few common and well established approaches. Most popular is the so called V-model defined in “ISO/IEC/IEEE 12207 Systems and software engineering - Software life cycle processes“ which can be depicted as follows:
The conclusion from above picture is that software development can be split into two major parts:
- design and implementation,
- verification (testing).
As we can see, the verification/testing part consists of many different test approaches/stages which are targeted to different stages of software creation. One of them is the topic of this article which is unit testing.
More about various SDLC methodologies can be found in the linked article.
Why is unit testing important?
But why should we test at all? This dilemma was discussed many times and I believe that most of the people involved in software development understand the importance of testing. Hardware driven by code designed by other people, nowadays AI also takes part in it, becoming an inseparable part of our life. It is hard to imagine what kind of crisis could emerge if a major part of this running software breaks. We have planes, nuclear power plants, life sustaining devices in hospitals, we store our money in banks which also use software applications, IoT devices, cell phones etc. There were many cases in the past when a small software bug caused a major financial loss or people lost their life.
One well-known example of a software error in the medical care system was a bug in Therac-25 radiation therapy machine 
used in cancer treatment in the 1980s.
It was designed to deliver both electron and photon beams, and was marketed as being safer and more efficient than earlier models. However, a series of accidents involving the Therac-25 resulted in several patients receiving massive overdoses of radiation, leading to serious injuries and in some cases, death .
Another example of software quality importance was the Heartbleed bug. The Heartbleed Bug was a flaw in the OpenSSL cryptographic software library which was used to secure many websites and online services. The bug allowed hackers to access sensitive information such as passwords and other confidential data that was meant to be encrypted.
As a result, many websites had to update their SSL certificates and passwords, which caused significant disruptions and financial losses for businesses. It was estimated that the bug affected around 17% of all secure web servers on the internet.
We can find many more examples when software bugs caused serious damage and financial losses. Another important fact about software bugs and testing is to catch problems as early as possible because of:
- cost reduction,
- quality improvement,
- risk management (early testing helps identify potential risks and vulnerabilities),
- faster time-to-market.
If the risk management topic matches your interests, check our previous article, where we talk more about risk management importance.
The Systems Sciences Institute at IBM has reported that “the cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase.” Another analysis can be found here .
After requirements analysis, system design, architecture design and module design steps review, software development starts and unit testing is the first line of defense to catch the problems.
In summary important benefits from (early) unit testing are:
- Early defect detection – unit testing enables developers to catch defects early in the software development process, before the code is integrated with other components. This can save significant time and cost by preventing defects from propagating to other parts of the system and reducing the complexity of debugging.
- Improved code quality– unit testing helps ensure that the code is reliable, maintainable, and scalable by enabling developers to identify and fix defects and design flaws before the code is integrated with other components.
- Facilitates refactoring – refactoring is a process of improving the design of the code without changing its external behavior. Unit tests help ensure that the code behaves as expected after refactoring, enabling developers to make changes with confidence. This is true if unit tests implementation is not tightly coupled with code, otherwise big refactoring of code requires a lot of changes in unit tests.
- Faster development cycles – unit testing helps reduce the time required to develop and maintain software by enabling developers to catch defects early in the process and reducing the time required for manual testing.
- Better documentation – unit tests serve as a form of documentation for the code, providing examples of how the code is expected to behave and helping new developers understand the codebase.
- Allows more frequent releases – by testing code in isolation, developers can quickly identify and fix issues, allowing for faster iteration and more frequent releases.
- Improves coding good practices – writing the code and unit tests for that helps to understand how to split the code into smaller functional and modular pieces.
So these are reasons which answer why we should test and why having a good suite of unit tests is beneficial but how should we approach unit testing at all? Are there techniques to write unit tests? Are there tools which can support us in writing unit tests?
How to write a unit test?
A unit test is a type of automated test that focuses on testing a single unit of functionality in a software system. The unit being tested is typically a single function, class member, or a small set of functions or methods, that perform a specific task or implement a specific feature.
The purpose of a unit test is to ensure that the unit being tested behaves as expected, under a range of input values and boundary conditions. Unit tests are also part of the test-driven development process and they are designed to catch defects early in the software development process, before the code is integrated with other components or dependencies.
To write a unit test a test designer should:
-
Identify “units” of code which can isolated and called:
- Single function,
- Method,
- Something which performs a specific task or implements specific feature.
-
Understand what is the function of that unit (what the unit is supposed to do, what is its purpose).
-
Find out what kind of input the unit takes and what kind of result is expected:
- No input at all?
- If there is input what kind of data it accepts and what kind of data is incorrect, not accepted?
- Does it return anything or maybe it should affect the system in some other way?
- Are there any boundaries or limitations for data?
- Identify what kind of input is considered incorrect and how the unit should handle incorrect input?
-
Understand what is positive and negative result from a unit.
-
Identify test data the feed the suite:
- There should be sets of data which will produce all possible ranges/classes of results or actions for a unit, positive or negative.
- Input test data should cover edge cases and invalid inputs.
What should good unit tests be?
Fast and atomic
-
Individual test cases should have one or two assertions at most, tests are atomic. If combined with test setup/teardown this approach guarantees better visibility of which parts of code works and which fails, otherwise a first assertion in a single test case will omit next checks in the same test. It provides better test coverage.
-
The individual unit test take a matter of milliseconds, allowing for a few tests to be completed per second.
-
In tests avoid interactions with other components/systems as much as possible, use mocks, patches to achieve this. This will reduce time of execution and test stability. Some advices:
- Don’t talk to a database.
- Don’t communicate across a network.
- Don’t touch the file system.
- Don’t rely on special environment settings (such as configuration files).
-
Fast and atomic tests count even more when there are a big number of test cases.
-
Having faster feedback allows us to fix bugs earlier.
-
Having atomic tests allows us to target the problem more precisely.
Independent
- Execution of one test should not affect next tests.
- Test setup, test teardown: each test should begin in a “default” state of a software being tested and should do cleanup actions when it ends especially when it failed and software was left in undefined state. Next test can’t rely on the actions performed by previous tests. It is a best practice to avoid assuming that tests will always be executed in the same order. Furthermore, we cannot predict when a single test may fail, potentially impacting the outcome of all other remaining tests. This approach improves test stability with repeatable results.
- Use mocks and patches to mimic external dependencies like databases, APIs, files, connections etc. If a unit test should check if software can extract or modify data from/in a database the test should focus on a unit being tested not a database itself as this is not the topic of the test. Reduce dependency from external components to improve time of execution and test stability. API can be down, database could have been modified, connection may not be opened because of network problems. It is possible to dump part of the database, save a short part of a file for testing or mimic network connection with other tools or implement a mock for it if needed.
Good unit test suite properties
-
Test suite is modular and maintainable: just like writing regular code, test suite implementation should follow good coding practices. If there are tests which have common parts extract it to a separate module/function. Object oriented programming approach can be used to classify different test groups, use abstract classes/inheritance to avoid code duplication. There may be common parts like suite setup, suite teardown etc.
Functional programming approach can also be applied: fixtures in PyTest can be an example here. -
Each test should have a meaningful name, it should describe what it tests, the output of the test result should also contain enough information to identify what and why exactly failed.
-
Tests should be grouped into test suites. It is possible to group tests in different ways considering different properties of each test. Test can be grouped with use of:
- Tags,
- Marks,
- Group by classes,
- Any mechanism which will allow you to run specific groups of tests with you test runner.
Test can be grouped by:
- Execution time: short, long,
- Test importance/priority,
- Test environment requirements: some tests can be run on specific hardware type,
- By functionality: add items to the cart, remove items from the cart…
Grouping tests allows us to run tests in a more precise and adaptive way. It may turn out there is a need to run critical only tests due to limited time constraints. It is more wise to run “fasts” tests first to have quicker feedback.
- Unit test suite should be automated and integrated in the build system. A suite of unit tests should be a first step before passing the software for further testing. If there is a CI/CD process performing unit tests should be integrated into it. In general tests should be executed as often as possible, if we lack CI/CD other approaches can be used like hooks in GIT for every code push or pull request.
Test suite has good code coverage
When writing tests take care to exercise code being tested in many ways considering different measures:
- Code coverage – this measures the percentage of code that has been executed during testing.
- Functional coverage – this measures the degree to which the functionality of the system or application has been tested.
- Statement coverage – this measures the percentage of individual statements in the code that have been executed during testing.
- Branch coverage – this measures the percentage of branches in the code that have been executed during testing.
- Path coverage – this measures the percentage of possible paths through the code that have been executed during testing.
- Boundary coverage – this measures the degree to which the boundary values of inputs have been tested.
- Error handling coverage – this measures the degree to which error handling and recovery paths have been tested. Test positive and negative scenarios.
Reasons to measure test code coverage
- Identify untested code – test code coverage helps to identify parts of the code that have not been tested.
- Measure testing effectiveness – test code coverage is a metric that can be used to measure the effectiveness of testing. It provides information about how much of the code has been tested and how much is still untested.
- Compliance requirements – test code coverage may be required for compliance reasons. For example, some industries such as healthcare or aerospace may require a certain level of test code coverage as part of their regulatory requirements.
- Confidence in changes – test code coverage can give developers and stakeholders confidence that changes to the code have not broken any existing functionality. By running tests after a change, developers can ensure that the code continues to function as expected.
Experience and knowledge in test practice and theory is valuable here but there are also tools which can support a test designer to analyze coverage and prepare appropriate data to increase coverage. Examples of Python tools for measuring test coverage:
- Coverage.py : statement, branch, path coverage
- pytest-cov : statement, branch, path coverage
- nose : statement, branch and condition coverage
- figleaf : statement and branch coverage
On the other hand there are tools which can be useful to prepare input data for tests like Python’s Hypothesis framework.
Summary
Unit testing is a crucial part of software development as it helps catch defects early in the process, reduce costs, improve code quality, and facilitate refactoring. To write effective unit tests, one should identify units of code and understand their purpose, input, and expected output. Tests should be fast, atomic, independent, and maintainable, and should cover different types of code coverage. Tools such as coverage.py, pytest-cov, and nose can assist in measuring and analyzing test coverage. A unit test suite should also be automated, take advantage of integrating it into the build system.
Furthermore, an effective unit test suite should have sufficient coverage, be grouped by functionality, importance/priority, and execution time. It's important to use mocks and patches to reduce dependency on external components and improve test stability. To achieve early feedback unit testing should be integrated into the development process and continuous integration/continuous deployment pipelines. By adhering to these best practices, developers can ensure that their software is more reliable, maintainable, and scalable.
