Python Project Structure & Module Architecture
Design well-organized Python projects with clear module boundaries, explicit public interfaces, and maintainable directory structures. Good organization makes code discoverable and changes predictable.
When to Use This Skill
- Starting a new Python project from scratch
- Reorganizing an existing codebase for clarity
- Defining module public APIs with
__all__
- Deciding between flat and nested directory structures
- Determining test file placement strategies
- Creating reusable library packages
Core Concepts
1. Module Cohesion
Group related code that changes together. A module should have a single, clear purpose.
2. Explicit Interfaces
Define what's public with __all__. Everything not listed is an internal implementation detail.
3. Flat Hierarchies
Prefer shallow directory structures. Add depth only for genuine sub-domains.
4. Consistent Conventions
Apply naming and organization patterns uniformly across the project.
Quick Start
myproject/
โโโ src/
โ โโโ myproject/
โ โโโ __init__.py
โ โโโ services/
โ โโโ models/
โ โโโ api/
โโโ tests/
โโโ pyproject.toml
โโโ README.md
Fundamental Patterns
Pattern 1: One Concept Per File
Each file should focus on a single concept or closely related set of functions. Consider splitting when a file:
- Handles multiple unrelated responsibilities
- Grows beyond 300-500 lines (varies by complexity)
- Contains classes that change for different reasons
Pattern 2: Explicit Public APIs with __all__
Define the public interface for every module. Unlisted members are internal implementation details.
from .user_service import UserService
from .order_service import OrderService
from .exceptions import ServiceError, ValidationError
__all__ = [
"UserService",
"OrderService",
"ServiceError",
"ValidationError",
]
Pattern 3: Flat Directory Structure
Prefer minimal nesting. Deep hierarchies make imports verbose and navigation difficult.
# Preferred: Flat structure
project/
โโโ api/
โ โโโ routes.py
โ โโโ middleware.py
โโโ services/
โ โโโ user_service.py
โ โโโ order_service.py
โโโ models/
โ โโโ user.py
โ โโโ order.py
โโโ utils/
โโโ validation.py
# Avoid: Deep nesting
project/core/internal/services/impl/user/
Add sub-packages only when there's a genuine sub-domain requiring isolation.
Pattern 4: Test File Organization
Choose one approach and apply it consistently throughout the project.
Option A: Colocated Tests
src/
โโโ user_service.py
โโโ test_user_service.py
โโโ order_service.py
โโโ test_order_service.py
Benefits: Tests live next to the code they verify. Easy to see coverage gaps.
Option B: Parallel Test Directory
src/
โโโ services/
โ โโโ user_service.py
โ โโโ order_service.py
tests/
โโโ services/
โ โโโ test_user_service.py
โ โโโ test_order_service.py
Benefits: Clean separation between production and test code. Standard for larger projects.
Advanced Patterns
Pattern 5: Package Initialization
Use __init__.py to provide a clean public interface for package consumers.
"""MyPackage - A library for doing useful things."""
from .core import MainClass, HelperClass
from .exceptions import PackageError, ConfigError
from .config import Settings
__all__ = [
"MainClass",
"HelperClass",
"PackageError",
"ConfigError",
"Settings",
]
__version__ = "1.0.0"
Consumers can then import directly from the package:
from mypackage import MainClass, Settings
Pattern 6: Layered Architecture
Organize code by architectural layer for clear separation of concerns.
myapp/
โโโ api/ # HTTP handlers, request/response
โ โโโ routes/
โ โโโ middleware/
โโโ services/ # Business logic
โโโ repositories/ # Data access
โโโ models/ # Domain entities
โโโ schemas/ # API schemas (Pydantic)
โโโ config/ # Configuration
Each layer should only depend on layers below it, never above.
Pattern 7: Domain-Driven Structure
For complex applications, organize by business domain rather than technical layer.
ecommerce/
โโโ users/
โ โโโ models.py
โ โโโ services.py
โ โโโ repository.py
โ โโโ api.py
โโโ orders/
โ โโโ models.py
โ โโโ services.py
โ โโโ repository.py
โ โโโ api.py
โโโ shared/
โโโ database.py
โโโ exceptions.py
File and Module Naming
Conventions
- Use
snake_case for all file and module names: user_repository.py
- Avoid abbreviations that obscure meaning:
user_repository.py not usr_repo.py
- Match class names to file names:
UserService in user_service.py
Import Style
Use absolute imports for clarity and reliability:
from myproject.services import UserService
from myproject.models import User
from ..services import UserService
from . import models
Relative imports can break when modules are moved or reorganized.
Best Practices Summary
- Keep files focused - One concept per file, consider splitting at 300-500 lines (varies by complexity)
- Define
__all__ explicitly - Make public interfaces clear
- Prefer flat structures - Add depth only for genuine sub-domains
- Use absolute imports - More reliable and clearer
- Be consistent - Apply patterns uniformly across the project
- Match names to content - File names should describe their purpose
- Separate concerns - Keep layers distinct and dependencies flowing one direction
- Document your structure - Include a README explaining the organization