Python Packaging
Comprehensive guide to creating, structuring, and distributing Python packages using modern packaging tools, pyproject.toml, and publishing to PyPI.
When to Use This Skill
- Creating Python libraries for distribution
- Building command-line tools with entry points
- Publishing packages to PyPI or private repositories
- Setting up Python project structure
- Creating installable packages with dependencies
- Building wheels and source distributions
- Versioning and releasing Python packages
- Creating namespace packages
- Implementing package metadata and classifiers
Core Concepts
1. Package Structure
- Source layout:
src/package_name/ (recommended)
- Flat layout:
package_name/ (simpler but less flexible)
- Package metadata: pyproject.toml, setup.py, or setup.cfg
- Distribution formats: wheel (.whl) and source distribution (.tar.gz)
2. Modern Packaging Standards
- PEP 517/518: Build system requirements
- PEP 621: Metadata in pyproject.toml
- PEP 660: Editable installs
- pyproject.toml: Single source of configuration
3. Build Backends
- setuptools: Traditional, widely used
- hatchling: Modern, opinionated
- flit: Lightweight, for pure Python
- poetry: Dependency management + packaging
4. Distribution
- PyPI: Python Package Index (public)
- TestPyPI: Testing before production
- Private repositories: JFrog, AWS CodeArtifact, etc.
Quick Start
Minimal Package Structure
my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── src/
│ └── my_package/
│ ├── __init__.py
│ └── module.py
└── tests/
└── test_module.py
Minimal pyproject.toml
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "my-package"
version = "0.1.0"
description = "A short description"
authors = [{name = "Your Name", email = "[email protected]"}]
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
"requests>=2.28.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.0",
"black>=22.0",
]
Package Structure Patterns
Pattern 1: Source Layout (Recommended)
my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── .gitignore
├── src/
│ └── my_package/
│ ├── __init__.py
│ ├── core.py
│ ├── utils.py
│ └── py.typed # For type hints
├── tests/
│ ├── __init__.py
│ ├── test_core.py
│ └── test_utils.py
└── docs/
└── index.md
Advantages:
- Prevents accidentally importing from source
- Cleaner test imports
- Better isolation
pyproject.toml for source layout:
[tool.setuptools.packages.find]
where = ["src"]
Pattern 2: Flat Layout
my-package/
├── pyproject.toml
├── README.md
├── my_package/
│ ├── __init__.py
│ └── module.py
└── tests/
└── test_module.py
Simpler but:
- Can import package without installing
- Less professional for libraries
Pattern 3: Multi-Package Project
project/
├── pyproject.toml
├── packages/
│ ├── package-a/
│ │ └── src/
│ │ └── package_a/
│ └── package-b/
│ └── src/
│ └── package_b/
└── tests/
Complete pyproject.toml Examples
Pattern 4: Full-Featured pyproject.toml
[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "my-awesome-package"
version = "1.0.0"
description = "An awesome Python package"
readme = "README.md"
requires-python = ">=3.8"
license = {text = "MIT"}
authors = [
{name = "Your Name", email = "[email protected]"},
]
maintainers = [
{name = "Maintainer Name", email = "[email protected]"},
]
keywords = ["example", "package", "awesome"]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
dependencies = [
"requests>=2.28.0,<3.0.0",
"click>=8.0.0",
"pydantic>=2.0.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.0.0",
"pytest-cov>=4.0.0",
"black>=23.0.0",
"ruff>=0.1.0",
"mypy>=1.0.0",
]
docs = [
"sphinx>=5.0.0",
"sphinx-rtd-theme>=1.0.0",
]
all = [
"my-awesome-package[dev,docs]",
]
[project.urls]
Homepage = "https://github.com/username/my-awesome-package"
Documentation = "https://my-awesome-package.readthedocs.io"
Repository = "https://github.com/username/my-awesome-package"
"Bug Tracker" = "https://github.com/username/my-awesome-package/issues"
Changelog = "https://github.com/username/my-awesome-package/blob/main/CHANGELOG.md"
[project.scripts]
my-cli = "my_package.cli:main"
awesome-tool = "my_package.tools:run"
[project.entry-points."my_package.plugins"]
plugin1 = "my_package.plugins:plugin1"
[tool.setuptools]
package-dir = {"" = "src"}
zip-safe = false
[tool.setuptools.packages.find]
where = ["src"]
include = ["my_package*"]
exclude = ["tests*"]
[tool.setuptools.package-data]
my_package = ["py.typed", "*.pyi", "data/*.json"]
[tool.black]
line-length = 100
target-version = ["py38", "py39", "py310", "py311"]
include = '\.pyi?$'
[tool.ruff]
line-length = 100
target-version = "py38"
[tool.ruff.lint]
select = ["E", "F", "I", "N"
