63.5 Parametrize: Running the Same Test With Multiple Inputs
Right, let’s talk about one of the most powerful tools in your testing arsenal: parametrization. You’ve written a test. It works. You feel a small, righteous glow of accomplishment. Then you realize you need to test the same function not just with one input, but with five. Or twenty. Your first instinct might be to copy-paste that test function a bunch of times, changing the input and expected output each time. Don’t. I’ve been there, and it’s a path that leads to madness, despair, and a test suite that’s a nightmare to maintain.
Parametrization is the civilized alternative. It’s the way you tell your test framework, “Hey, run this exact same test logic, but for each of these different input/output combinations.” It’s DRY (Don’t Repeat Yourself) principle applied to testing, and it’s a beautiful thing.
The @pytest.mark.parametrize Decoration
pytest handles this with elegance via a decorator. The syntax looks a bit odd at first, but you’ll get used to it. Here’s the classic, trivial example everyone uses, and for good reason—it’s perfectly clear.
# test_math.py
import pytest
def test_add_two_numbers(a, b, expected):
assert a + b == expected
@pytest.mark.parametrize("a, b, expected", [
(1, 2, 3),
(5, -5, 0),
(3.14, 2.86, 6.0),
("Hello, ", "world!", "Hello, world!"), # Wait, can we add strings? Yes, we can.
])
def test_add_two_numbers(a, b, expected):
assert a + b == expected
Run this with pytest -v, and you’ll see glorious output like:
test_math.py::test_add_two_numbers[1-2-3] PASSED
test_math.py::test_add_two_numbers[5--5-0] PASSED
test_math.py::test_add_two_numbers[3.14-2.86-6.0] PASSED
test_math.py::test_add_two_numbers[Hello, -world!-Hello, world!] PASSED
See what happened? pytest didn’t run one test. It ran four distinct tests, each with a unique ID generated from its parameters. If the third one had failed, it would tell you exactly which set of inputs caused the problem. This is infinitely better than a test called test_add_two_numbers that fails and leaves you guessing which scenario broke.
The unittest Equivalent: subTest
Now, what if you’re stuck in a unittest prison? The framework’s design is a bit more verbose, but it has a solid answer: subTest(). It’s not as syntactically sweet as a decorator, but it gets the job done.
# test_math_unittest.py
import unittest
class TestMath(unittest.TestCase):
def test_add_two_numbers(self):
test_cases = [
(1, 2, 3),
(5, -5, 0),
(3.14, 2.86, 6.0),
]
for a, b, expected in test_cases:
with self.subTest(a=a, b=b, expected=expected):
self.assertEqual(a + b, expected)
The key here is the with self.subTest(...) context manager. Inside that block, if an assertion fails, the test will report the failure but keep going with the other cases. It will then tell you precisely which subtest failed and with which parameters. It’s clunkier than pytest because you’re writing a loop yourself, but the functionality is there.
Why This is More Than Just Convenience
This isn’t just about saving keystrokes. It’s about creating a maintainable and clear specification.
- The Test is the Specification: A well-parametrized test acts as a living document of the function’s behavior under various conditions. Looking at the list of parameters shows you the happy path, the edge cases, and the weird inputs you decided to handle (or not handle) all in one place.
- Isolation of Failures: As I mentioned, a failure in one set of parameters doesn’t block the execution of the others. You get a full report of what’s working and what’s broken, which is invaluable for debugging.
- Avoids Test Interference: Using separate test invocations ensures that each run has a fresh setup and teardown. A loop inside a single test could accidentally let state leak from one case to another, creating bizarre, hard-to-reproduce bugs in your tests. Parametrization avoids this entirely.
Common Pitfalls and How to Avoid Them
Of course, this power comes with a few ways to shoot yourself in the foot.
The Giant Blob of Data: It’s tempting to create a massive list of 50 test cases. Don’t. It becomes unreadable. If you have a huge matrix of inputs, generate them programmatically inside the test file or, even better, load them from a separate JSON or CSV file. Keep the parametrize decorator for the most illustrative and critical examples.
Mixing Unrelated Concerns: Parametrization is for testing the same behavior with different inputs. If you find yourself trying to test different behaviors (e.g., testing addition and exception raising), you’ve gone too far. Split that into separate test functions. One test, one behavior.
The Unreadable Test ID: Look at the string example in our first code block. The test ID became
[Hello, -world!-Hello, world!]. That’s… a mess. For complex parameters (like objects, long strings, etc.), use theidsargument to provide a clear, human-readable label for each test case.def id_func(test_case): a, b, expected = test_case return f"add_{a}_and_{b}" @pytest.mark.parametrize("a, b, expected", [ (1, 2, 3), (5, -5, 0), ], ids=id_func) def test_add_two_numbers(a, b, expected): assert a + b == expectedNow your output will show
test_add_two_numbers[add_1_and_2]andtest_add_two_numbers[add_5_and_-5]. Much better.
The bottom line is this: any time you catch yourself writing a test and thinking “and it should also work for…”, you should immediately reach for parametrization. It transforms your tests from a simple pass/fail check into a robust, explicit definition of your code’s contract.