Python has become a popular choice among developers thanks to its versatility, readability, and vast library ecosystem. However, as with any programming language, these projects can become complicated, especially when you need to support multiple Python versions and various dependencies. This is where tox steps in as a powerful automation tool to streamline your project's testing, building, and deployment processes.
In this blog post, we'll explore what tox is, the problems it solves, its capabilities, how it's used to automate and standardize testing, and how you can integrate it into your CI/CD pipeline.
What is Python Tox?
Tox is a generic virtual environment management and test command line tool. It’s an open-source, command-line driven automation tool designed for Python projects. It helps developers automate various tasks, such as testing, building, packaging, and deployment across different environments. By managing virtual environments for each target version and configuration, tox solves a lot of common issues, ensuring your code maintains compatibility across a range of versions and varying dependency setups.
Tox supports a wide range of versions, from the older version 2.7 to the latest 3.x releases. This tool can even handle projects with multiple supported versions, making it an excellent choice for test libraries and applications designed to work with various language releases.
If you’re wondering what kind of projects can benefit from tox, learn what Python is used for from our article.
Tox for Python software
Tox is a versatile tool that goes beyond managing environments and dependencies. It offers the flexibility to work alongside any command-line tool, making it a powerful addition to your development workflow. This means that, in addition to language-specific tools, you can integrate a wide range of utilities that may be relevant to your project, such as code formatters, version control systems, or deployment tools. This tool offers seamless integration with a variety of popular linting, code analysis, and test tools, allowing you to create a comprehensive test workflow for your projects. Here are some popular Python tools that work especially well with it.
Pytest
Pytest is a powerful and popular test tool that makes it easy to write and run tests. By configuring tox to use pytest test tool as the test runner, you can take advantage of pytest's features like fixtures, parameterized tests, and advanced reporting. You can find more top Python testing frameworks on our blog.
Flake8
Flake8 is a linter that checks your code for compliance with PEP 8, the language’s style guide, as well as for common programming errors. Integrating flake8 with tox allows you to enforce consistent coding standards across your project. If you’re interested in linters, check out our Best practices for Python code quality [link].
Mypy
Mypy is a popular static type checker that helps you catch potential type errors before they become runtime issues. By integrating mypy with tox, you can ensure that your code adheres to the type hints and annotations you've specified, improving the overall reliability and maintainability of your project.
Coverage
Coverage is a tool for measuring code coverage in your projects. By using coverage with tox, you can generate detailed reports on how well your tests cover your application's code, making it easier to identify areas that need more testing. And if you're interested in using Python for network test automation, read our previous article.
Sphinx
Sphinx is a powerful documentation generator. You can configure tox to build your project's documentation using Sphinx, ensuring that your documentation is always up-to-date and compatible with the latest code changes.
By combining these tools, you can create a powerful and flexible test workflow for your projects that goes beyond simple compatibility testing.
Streamlining project management with tox
Tox aims to simplify project management by addressing common challenges, such as compatibility, reproducibility, and efficient testing. It creates and manages a virtual environment for each target language version and configuration. By default, this tool uses the virtualenv package to create these isolated environments, but it can also be configured to use alternatives like poetry or conda. Each environment is created with the specified language version and dependencies, ensuring that tests are run in a clean and isolated environment that matches the target configuration. This prevents issues arising from shared dependencies or system-wide configurations.
Tox solves the following challanges:
- Compatibility: these projects often need to support different Python versions and dependency configurations, which can be a challenging task. Tox makes it easier to manage these complexities by automating the test process across different environments.
- Reproducibility: tox creates isolated test environments, ensuring that tests are run with consistent settings and dependencies, making it easier to reproduce and debug issues.
- Efficient testing: tox allows developers to run tests in parallel, speeding up the process and making it more efficient.
With tox managing your environment and addressing these common challenges, you can focus on writing great code and delivering high-quality applications.
Getting started with tox: installation, configuration, and execution
To get started with this tool, you'll need to install it, create a basic configuration file, and learn how to run it from the command line. This section will guide you through these steps.
Installing tox
Tox requires an interpreter of version 3.7 or higher. It can be easily installed using pipx, a tool for installing and running CLI applications in isolated environments. If you haven't installed pipx yet, you can install it with the following command:
python -m pip install pipx-in-pipx --user
After installing pipx, make sure to add the pipx binary directory to your PATH environment variable. You can find the directory using pipx ensurepath.
Now, you can install it with pipx by running the following command:
pipx install tox
Using pipx ensures that the tool is installed in an isolated environment, avoiding potential conflicts with other packages on your system.
For other supported installation methods refer to official documentation 
.
Creating a basic configuration for tox and project setup
After tox installs correctly, you'll need to create a configuration file called tox.ini in your project's root directory. This file will contain the settings for your target language versions, dependencies, and test commands.
Here's an example of a basic tox.ini configuration:
[tox]
envlist = py37, py38, py39
[testenv]
deps =
pytest
-rrequirements.txt
commands =
pytest tests/
In this example, we've specified three target versions (3.7, 3.8, and 3.9) in the envlist setting. The [testenv] section defines the dependencies and commands to be executed for each environment. We've specified pytest as a dependency, and the -rrequirements.txt line indicates that the project's dependencies should also be installed from the requirements.txt file. The commands setting makes tox run pytest on the tests/ directory.
Running tox from the command line
With the tool installed and the configuration file in place, you can now run tox from your project's root directory using the command line. To run each environment specified in the tox.ini file, simply type the following command:
tox
To execute only a specific environment, you can use the -e flag followed by the environment name. For example, to run tests only for version 3.8, you can use the following command:
tox -e py38
It also allows you to run environments in parallel, which can significantly speed up the testing process. To do this you can add the --parallel flag:
tox --parallel
Running tox for packaging
To run tox for packaging, you need to add a new environment dedicated to building the distribution package. In your tox.ini file, add a new environment called build:
[tox]
envlist = py37, py38, py39, build
isolated_build = true
[testenv]
deps =
pytest
-rrequirements.txt
commands =
pytest tests/
[testenv:build]
basepython = python3.9
skip_install = true
deps =
setuptools
wheel
commands =
python setup.py sdist bdist_wheel
In this example, the basepython option is set to python3.9 within the [testenv:build] section. This informs tox to use version 3.9 for the build environment. Make sure that the specified version installs correctly on your system, or else it will report an error.
The [testenv:build] section defines the build environment with the skip_install option set to true to skip the installation of the package in the test environment. The setuptools and wheel dependencies are added to create the source distribution (sdist) and wheel distribution (bdist_wheel). The commands setting tells tox to run setup.py with the sdist and bdist_wheel options to build both types of packages.
Advanced tox configuration for comprehensive testing and building
In this section, we will explore an advanced tox.ini configuration that leverages the full potential of the tool to perform comprehensive testing, code style checking, type checking, and documentation building for your project. The advanced configuration is designed to provide a robust testing and building setup that ensures your project maintains a high level of quality and compatibility across many versions and platforms of the language. By the end of this section, you will have a deeper understanding of the advanced features it offers and how to configure them to suit your project's requirements.
[tox]
envlist = py{37,38,39}-{linux,macos,win}, flake8, mypy, docs, build
isolated_build = true
minversion = 3.7
[testenv]
platform =
linux: linux
macos: darwin
win: win32
deps =
pytest
pytest-cov
-rrequirements.txt
commands =
pytest --cov=src/ --cov-report=term-missing tests/
[testenv:flake8]
basepython = python3.9
skip_install = true
deps =
flake8
commands =
flake8 src/
[testenv:mypy]
basepython = python3.9
skip_install = true
deps =
mypy
commands =
mypy src/
[testenv:docs]
basepython = python3.9
skip_install = true
deps =
Sphinx
sphinx_rtd_theme
commands =
sphinx-build -b html docs/ docs/_build/
[testenv:build]
basepython = python3.9
skip_install = true
deps =
setuptools
wheel
commands =
python setup.py sdist bdist_wheel
[flake8]
exclude = .tox,*.egg,build,data
select = E,W,F
max-line-length = 120
This configuration includes:
- Multiple test environments with different versions and platforms. The envlist now includes py{37,38,39}-{linux,macos,win} to test across versions 3.7, 3.8, and 3.9 of the language on Linux, macOS, and Windows platforms.
- The platform option is used within the [testenv] section to specify which platform each environment should run on.
- A code coverage report is generated using the pytest-cov package while running the tests.
- Additional environments for code style checking with flake8, type checking with mypy, and building the documentation using Sphinx.
- The basepython option is specified for each additional environment to ensure that version 3.9 is used for those tasks.
- The skip_install option is used for additional environments where the package installation is not needed.
- The minversion option is set to 3.7 in the [tox] section to ensure that at least version 3.7 is available for running tox.
- A [flake8] section is added to specify flake8-related settings, such as excluded directories, selected error codes, and maximum line length.
This configuration demonstrates the flexibility and power of the tool, allowing you to run tests, checks, and build documentation across multiple versions and platforms, ensuring your project's compatibility and quality.
Integrating tox in CI/CD
Integrating this tool into your Continuous Integration (CI) and Continuous Deployment (CD) pipeline can greatly streamline your development workflow by automating the testing and building process whenever new code is pushed to the repository. By doing so, you can quickly detect and address any issues or regressions before they reach production.
Here's an overview of how to integrate it with Gitlab CI/CD.
To integrate tox with GitLab CI/CD, create a .gitlab-ci.yml file in your repository's root directory. Configure the GitLab CI/CD pipeline to run tox as follows:
stages:
- lint
- test
- build
- docs
# Common stage template
.common_template:
before_script:
- python -m pip install --upgrade pip
- pip install tox
cache:
paths:
- .tox/
key: "${CI_COMMIT_REF_SLUG}-${CI_JOB_NAME}-${CI_COMMIT_SHA}"
# Linting
lint:
stage: lint
image: python:3.9
script:
- tox -e flake8,mypy
extends: .common_template
# Testing
test:python37:
stage: test
script:
- tox -e py37
extends: .common_template
image: python:3.7
test:python38:
stage: test
script:
- tox -e py38
extends: .common_template
image: python:3.8
test:python39:
stage: test
script:
- tox -e py39
extends: .common_template
image: python:3.9
# Building
build:
stage: build
image: python:3.9
script:
- tox -e build
artifacts:
paths:
- dist/
extends: .common_template
# Documentation
docs:
stage: docs
image: python:3.9
script:
- tox -e docs
artifacts:
paths:
- docs/_build/
extends: .common_template
This configuration file defines four stages: lint, test, build, and docs. The lint stage runs flake8 for code style checking. The test stage uses a template and extends it for each version, running tests in parallel. The build stage creates a distribution package, and the docs stage builds the project documentation.
The .tox folder is cached in each job to speed up subsequent runs by reusing the virtual environment and installed dependencies. The cache key is based on the commit reference slug, job name, and commit SHA. The cache will be reused if the key matches, otherwise, a new cache entry will be created.
Artifacts are used to store the output of the build and docs stages. The distribution packages are stored under the dist/ directory, and the built documentation is stored under the docs/_build/ directory.
Conclusion
Tox is a powerful and flexible automation tool for Python projects that can help streamline your testing, building, and release process. By integrating it into the process of Python software testing and CI/CD pipeline, you can ensure that your applications are compatible with different language versions or installation dependencies. Embracing this tool can help you improve the quality and reliability of your projects.
