The requirements.txt file is the cornerstone of dependency management and environment reproducibility in the Python ecosystem. It serves as a manifest, a precise record of the exact package versions a project needs to function correctly. Its primary purpose is to freeze the state of an environment so that it can be perfectly reconstructed on another machine, whether that’s a colleague’s laptop, a CI/CD server, or a production deployment. This practice eliminates the infamous “but it works on my machine” problem by ensuring all collaborators and systems use identical dependency trees.

Generating requirements.txt with pip freeze

The most common method for generating a requirements.txt file is the pip freeze command. This command queries the current environment’s pip installation and outputs a list of all installed packages and their exact, pinned versions. This precision is crucial for reproducibility.

# Activate your project's virtual environment first
source venv/bin/activate
# On Windows: venv\Scripts\activate

# Install the packages your project needs
pip install requests django pandas

# Then, freeze the state of the environment to a file
pip freeze > requirements.txt

The resulting requirements.txt file will contain entries like:

certifi==2023.11.17
charset-normalizer==3.3.2
Django==5.0.1
idna==3.6
numpy==1.26.2
pandas==2.1.3
python-dateutil==2.8.2
pytz==2023.3.post1
requests==2.31.0
six==1.16.0
tzdata==2023.4

It is a critical best practice to always generate requirements.txt from within an activated, clean virtual environment that contains only the dependencies your project directly needs. Freezing from a global Python environment will include dozens of unnecessary and potentially conflicting packages, polluting your project’s dependency specification.

Installing from a requirements.txt File

To recreate the environment on another system, you simply use pip install with the -r flag. This command reads the file and installs every listed package at the specified version.

# Create and activate a new virtual environment first
python -m venv venv
source venv/bin/activate

# Install all dependencies from the requirements file
pip install -r requirements.txt

pip will resolve and install the entire dependency graph, ensuring the resulting environment is a functional clone of the original. This is the standard workflow for onboarding new developers and deploying applications.

The Pitfalls of Over-pinning and pip freeze

While pip freeze is excellent for reproducibility, it has a significant downside: it pins everything, including sub-dependencies (e.g., certifi, six, tzdata). This can lead to a bloated and overly restrictive requirements.txt file. If a top-level package like pandas releases an update that uses newer, compatible versions of its sub-dependencies, your requirements.txt will prevent you from installing that update because it explicitly demands the old versions of numpy or python-dateutil. This can hinder security updates and general maintenance.

Best Practices for a Maintainable requirements.txt

To avoid the “over-pinning” problem, a more nuanced approach is recommended for long-term maintainability. The key is to separate direct dependencies from indirect ones.

  1. Manual Curation for Direct Dependencies: Manually maintain a list of your project’s direct dependencies, preferably with loose version specifiers that allow for non-breaking updates. This is often done in a setup.py or pyproject.toml file, but can also be a hand-written requirements.txt.

    # This is a hand-crafted requirements.txt for direct dependencies
Django>=5.0,<5.1  # Pinned to the current major release
requests>=2.28.0,<3.0.0  # Allows any non-breaking 2.x update
pandas~=2.1.0  # Allows any 2.1.x release (bug fixes)

  2. Using requirements.txt for Full Reproduction: Use pip freeze > requirements.txt to generate the full, pinned list for deployment and strict reproduction. This file is a snapshot, not a policy. You would regenerate this file whenever you deliberately update your dependencies.

  3. The Two-File Approach: A common pattern is to have two files:

    • requirements.in: A hand-edited file containing only your direct dependencies with version ranges.
    • requirements.txt: The fully pinned output generated by a tool like pip-compile from the pip-tools package. This tool resolves the requirements.in file and generates a complete requirements.txt with all sub-dependencies pinned.
    # First, create a requirements.in file with your direct deps
echo "Django>=5.0,<5.1" > requirements.in
echo "requests>=2.28.0,<3.0.0" >> requirements.in

# Then use pip-compile to resolve and pin all dependencies
pip-compile requirements.in
# This generates a requirements.txt with all packages pinned.

Hash Checking for Ultimate Reproducibility and Security

For critical deployments where security and absolute fidelity are paramount, you can generate a requirements.txt file with hashes. This instructs pip to verify that the downloaded package archives match the expected cryptographic hash, ensuring that a compromised package index or a man-in-the-middle attack cannot inject malicious code.

pip freeze --all | pip hash > requirements.txt

Or, more commonly with pip-compile:

pip-compile requirements.in --generate-hashes

This will add a --hash=sha256:... flag to each line. When installing with pip install -r requirements.txt, pip will fail if any downloaded package does not match one of the supplied hashes.