84.5 tox: Testing Across Multiple Python Versions
Right, so you’ve written some tests. Good for you. But are you running them against the same old Python version you’re developing on? That’s like a chef only tasting their own food—of course it tastes good to you. The real world is a messy place full of different Python environments, and your code needs to work in all of them. Enter tox, the conductor of this particular orchestra of chaos. It’s not a test runner itself; it’s the automation tool that creates isolated environments, installs your stuff, and runs your chosen test runner (like pytest) across multiple Python versions. It’s the “it works on my machine” exterminator.
Why You Bother With This At All
Think of tox as your dedicated quality assurance team trapped inside a config file. Its primary job is to guarantee consistency. By forcing your tests to pass in a pristine, version-specific environment that tox creates from scratch every time (or cleverly reuses), you eliminate the “works on my machine” fallacy. You’re testing against a clean slate, just like your users will experience when they pip install your package. It also neatly handles testing under different dependency matrices—like making sure your library works with both the latest pandas and the oldest version you claim to support.
The Heart of the Beast: tox.ini
Everything tox does is driven by the tox.ini file in your project root. This is where you lay down the law. A basic one looks something like this:
# tox.ini
[tox]
envlist = py38, py39, py310, py311
[testenv]
deps =
pytest
pytest-cov
commands =
pytest tests/ --cov=my_package
Let’s autopsy this. The [tox] section defines envlist: the list of environments tox will create and run. Here, it’s creating four for four different Python versions. The magic pyXX is a default factor tox understands.
The [testenv] section is the blueprint for every environment. deps lists what to pip install into the environment before running your commands. commands is what actually gets executed to run the tests. Here, it’s calling pytest directly.
Making It Actually Find Your Python Versions
Here’s the first “gotcha.” tox doesn’t magically install Python 3.8 for you. It expects to find these interpreters on your system PATH. If you run tox and it screams about not finding python3.8, you have two options:
- Install the missing versions manually. Use
pyenvon Unix or install from python.org on Windows. This is the “proper” way. - The Lazy (and Brilliant) Hack: Use the
tox-pyenvplugin. Install it (pip install tox-pyenv) and it teachestoxto talk topyenv, automatically finding and using the interpreters you have installed without you needing to mess with your system PATH. It’s a lifesaver.
The Real-World, Not-So-Basic Example
Your basic tox.ini is cute, but let’s get serious. You need to test against multiple dependency sets. This is where tox’s factorization feature shines. Let’s say you need to ensure compatibility with both Django 3.2 and 4.2.
# tox.ini
[tox]
envlist =
py{38,39}-django32,
py{310,311}-django{32,42}
[testenv]
deps =
django32: Django>=3.2,<3.3
django42: Django>=4.2,<4.3
pytest
commands =
pytest tests/
Boom. Look at that envlist. You’re now generating environments for:
py38-django32,py39-django32py310-django32,py310-django42py311-django32,py311-django42
The deps section uses conditionals. For any environment with the django32 factor, it installs that specific Django version. Same for django42. This is how you stop relying on whatever pip install django would grab by default and start testing explicitly.
Pitfalls and Sharp Edges
The Wheel Pain: Building a package with C extensions?
toxwill try to build it from source in each isolated environment. This is slow and can fail if you lack build tools. The fix: Always runpython -m buildto generate source and wheel distributions first, then runtox. It will find the pre-builtsdistand install it instantly, skipping the build step.tox -e py311 --installpkg dist/my_pkg-1.0.0.tar.gzis your friend for testing a specific build.The Missing Dependency: Your tests might rely on a system library (like
libpqforpsycopg2). The pristinetoxenvironment won’t have it. The fix: You’ll need to pre-install those system dependencies before runningtox. This is often handled by your CI system (e.g.,apt-get installin a GitHub Action step).It’s Slow: Recreating environments from scratch every time is safe but tedious. Use
tox --recreateonly when you need a truly clean slate (e.g., after changingdeps). For rapid iteration,toxwill reuse the environment, which is much faster. Usetox -e py311to run just one specific environment.
The bottom line? tox is the gatekeeper. If it passes, you can be genuinely confident your code works. It moves the pain of environment management from your users and your CI script to your local machine, which is exactly where it belongs. Stop guessing and start testing.