The pyproject.toml file, introduced in PEP 518, marks a fundamental shift in the Python packaging ecosystem. It replaces the patchwork of setup.py, setup.cfg, and MANIFEST.in files with a single, static, declarative configuration file. This shift is crucial for several reasons. First, it eliminates the need to execute arbitrary code (as was required with setup.py) during package installation, significantly enhancing security and predictability. Second, being declarative, it allows tools to read and understand project metadata without needing to install the package first. Finally, it provides a standardized location for configuration of all the tools in a project’s toolchain, not just packaging.

Core Project Metadata with [project]

The heart of a package’s definition lies in the [project] table. This is where you declare essential metadata that will be displayed on PyPI and used by installers like pip. The most critical entry is name, which must be unique on PyPI and contain only ASCII letters, numbers, underscores, and hyphens.

[project]
name = "acme-super-logger"
version = "1.0.0"
description = "A high-performance logging library for all your needs."
readme = "README.md"
requires-python = ">=3.8"
license = {file = "LICENSE.txt"}
authors = [
    {name = "Jane Developer", email = "jane@example.com"},
    {name = "John Coder", email = "john@example.com"}
]
keywords = ["logging", "debugging", "performance"]
classifiers = [
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Operating System :: OS Independent",
]
dependencies = [
    "requests >=2.25.0",
    "colorama; sys_platform == 'win32'"
]

The version field is best managed dynamically using a build-time dependency like setuptools-scm or by reading from a __version__.py file, as hardcoding it here can lead to maintenance headaches and missed releases. The dependencies list is where you specify the package’s direct runtime dependencies. Conditional dependencies, like the colorama example above which is only required on Windows, can be specified using environment markers.

Defining Package Discovery (for setuptools)

While pyproject.toml is standard, the actual process of building a distributable package (a .whl or .tar.gz file) is handled by a build backend. setuptools is the most common backend. To use it, you must declare it in the [build-system] table. For setuptools to know which directories to include in the package, you must configure package discovery within the [tool.setuptools] table.

[build-system]
requires = ["setuptools>=64.0.0", "wheel"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
# For simple project structures (packages under 'src' or the project root)
packages = {find = {}} # Automatically find packages

# For a 'src' layout (recommended for avoiding import confusion)
# packages = {find = {where = ["src"]}}

[tool.setuptools.package-data]
# Include non-Python files (e.g., templates, data files)
"acme_super_logger" = ["*.json", "templates/*.html"]

The [build-system] table is non-negotiable; it tells tools like pip and build what is needed to create your package. The packages configuration is critical—failing to define it properly is a common pitfall that results in your source code not being included in the built distribution, leading to an empty or broken package.

Specifying Optional Dependencies

Many packages offer optional features that require additional dependencies. These are defined in the [project.optional-dependencies] table. This allows users to install them on-demand using the pip extras syntax: pip install acme-super-logger[dev,test].

[project.optional-dependencies]
dev = [
    "pytest >=7.0.0",
    "black >=22.0.0",
    "mypy >=0.900",
    "sphinx >=4.0.0"
]
test = [
    "pytest >=7.0.0",
    "pytest-cov >=3.0.0"
]
performance = [
    "orjson >=3.0.0"
]

Entry Points and Scripts

To create command-line scripts or expose plugins for your package (e.g., for a framework like Flask or Pluggable), you use the [project.scripts] and [project.gui-scripts] tables. This is the modern replacement for the scripts keyword in setup.py. The build backend will create the appropriate console scripts for the target operating system.

[project.scripts]
# Creates a console script named 'acme-log'
acme-log = "acme_super_logger.cli:main"

[project.gui-scripts]
# For GUI applications on Windows
acme-log-gui = "acme_super_logger.gui:launch"

[tool.setuptools.packages.find]
where = ["src"]

A best practice is to point the entry point to a function (like main) rather than a module. This ensures the function is not executed on import, which improves startup performance for the command-line tool.

Dynamic Metadata

For values that cannot be statically defined, most notably version, you can use dynamic fields. You must then tell setuptools how to resolve these dynamic values by configuring the corresponding attr in the [tool.setuptools.dynamic] table. A very robust pattern is to store the version in a __version__.py file inside your package and read it from there.

[project]
name = "acme-super-logger"
dynamic = ["version"]
# ... other static metadata ...

[tool.setuptools.dynamic]
version = {attr = "acme_super_logger.__version__"}

The corresponding __version__.py file is simple and should be kept minimal:

# src/acme_super_logger/__version__.py
__version__ = "1.0.0"

This approach provides a single source of truth for the version that can also be accessed within your application’s code at runtime via import acme_super_logger; print(acme_super_logger.__version__).