Testing Patterns
A pragmatic approach to testing that emphasises:
- Live testing over mocks
- Agent execution to preserve context
- YAML specs as documentation and tests
- Persistent results committed to git
Philosophy
This is not traditional TDD. Instead:
- Test in production/staging with good logging
- Use agents to run tests (keeps main context clean)
- Define tests declaratively in YAML (human-readable, version-controlled)
- Focus on integration (real servers, real data)
Why Agent-Based Testing?
Running 50 tests in the main conversation would consume your entire context window. By delegating to a sub-agent:
- Main context stays clean for development
- Agent can run many tests without context pressure
- Results come back as a summary
- Failed tests get detailed investigation
Commands
| Command |
Purpose |
/create-tests |
Discover project, generate test specs + testing agent |
/run-tests |
Execute tests via agent(s), report results |
/coverage |
Generate coverage report and identify uncovered code paths |
Quick workflow:
/create-tests → Generates tests/specs/*.yaml + .claude/agents/test-runner.md
/run-tests → Spawns agent, runs all tests, saves results
/run-tests api → Run only specs matching "api"
/run-tests --failed → Re-run only failed tests
/coverage → Run tests with coverage, analyse gaps
/coverage --threshold 80 → Fail if below 80%
Getting Started in a New Project
This skill provides the pattern and format. Claude designs the actual tests based on your project context.
What happens when you ask "Create tests for this project":
-
Discovery - Claude examines the project:
- What MCP servers are configured?
- What APIs or tools exist?
- What does the code do?
-
Test Design - Claude creates project-specific tests:
- Test cases for the actual tools/endpoints
- Expected values based on real behavior
- Edge cases relevant to this domain
-
Structure - Using patterns from this skill:
- YAML specs in
tests/ directory
- Optional testing agent in
.claude/agents/
- Results saved to
tests/results/
Example:
You: "Create tests for this MCP server"
Claude: [Discovers this is a Google Calendar MCP]
[Sees tools: calendar_events, calendar_create, calendar_delete]
[Designs test cases:]
tests/calendar-events.yaml:
- list_upcoming_events (expect: array, count_gte 0)
- search_by_keyword (expect: contains search term)
- invalid_date_range (expect: error status)
tests/calendar-mutations.yaml:
- create_event (expect: success, returns event_id)
- delete_nonexistent (expect: error, contains "not found")
The skill teaches Claude:
- How to structure YAML test specs
- What validation rules are available
- How to create testing agents
- When to use parallel execution
Your project provides:
- What to actually test
- Expected values and behaviors
- Domain-specific edge cases
YAML Test Spec Format
name: Feature Tests
description: What these tests validate
defaults:
tool: my_tool_name
timeout: 5000
tests:
- name: test_case_name
description: Human-readable purpose
tool: tool_name
params:
action: search
query: "test input"
expect:
contains: "expected substring"
not_contains: "should not appear"
status: success
Validation Rules
| Rule |
Description |
Example |
contains |
Response contains string |
contains: "from:john" |
not_contains |
Response doesn't contain |
not_contains: "error" |
matches |
Regex pattern match |
matches: "after:\\d{4}" |
json_path |
Check value at JSON path |
json_path: "$.results[0].name" |
equals |
Exact value match |
equals: "success" |
status |
Check success/error |
status: success |
count_gte |
Array length >= N |
count_gte: 1 |
count_eq |
Array length == N |
count_eq: 5 |
type |
Value type check |
type: array |
See references/validation-rules.md for complete documentation.
Creating a Testing Agent
Testing agents inherit MCP tools from the session. Create an agent that:
- Reads YAML test specs
- Executes tool calls with params
- Validates responses against expectations
- Reports results
Agent Template
CRITICAL: Do NOT specify a tools field if you need MCP access. When you specify ANY tools, it becomes an allowlist and "*" is interpreted literally (not as a wildcard). Omit tools entirely to inherit ALL tools from the parent session.
---
name: my-tester
description: |
Tests [domain] functionality. Reads YAML test specs and validates responses.
Use when: testing after changes, running regression tests.
model: sonnet
---
1. Find test specs: `tests/*.yaml`
2. Parse and execute each test
3. Validate responses
4. Report pass/fail summary
tests/
├── feature-a.yaml
├── feature-b.yaml
└── results/
└── YYYY-MM-DD-HHMMSS.md
For each test:
1. Call tool with params
2. Capture response
3. Apply validation rules
4. Record PASS/FAIL
Save results to `tests/results/YYYY-MM-DD-HHMMSS.md`
See templates/test-agent.md for complete template.
Results Format
Test results are saved as markdown for git history:
# Test Results: feature-name
**Date**: 2026-02-02 14:30
**Commit**: abc1234
**Summary**: 8/9 passed (89%)
## Results
- test_basic_search - PASSED (0.3s)
- test_with_filter - PASSED (0.4s)
- test_edge_case - FAILED
## Failed Test Details
### test_edge_case
- **Expected**: Contains "expected value"
- **Actual**: Response was empty
- **Params**: `{ action: search, query: "" }`
Save to: tests/results/YYYY-MM-DD-HHMMSS.md
Workflow
1. Create Test Specs
name: Search Tests
defaults:
tool: my_search_tool
tests:
- name: basic_search
params: { query: "hello" }
expect: { status: success, count_gte: 0 }
- name: filtered_search
params: { query: "hello", filter: "recent" }
expect: { contains: "results" }
2. Create Testing Agent
Copy templates/test-agent.md and customise for your domain.
3. Run Tests
"Run the search tests"
"Test the API after my changes"
"Run regression tests for gmail-mcp"
4. Review Results
Results saved to tests/results/. Commit them for history:
git add tests/results/
git commit -m "Test results: 8/9 passed"
Parallel Test Execution
Run multiple test agents simultaneously to speed up large test suites:
"Run these test suites in parallel:
- Agent 1: tests/auth/*.yaml
- Agent 2: tests/search/*.yaml
- Agent 3: tests/api/*.yaml"
Each agent:
- Has its own context (won't bloat main conversation)
- Can run 10-50 tests independently
- Returns a summary when done
- Inherits MCP tools from parent session
Why parallel agents?
- 50 tests in main context = context exhaustion
- 50 tests across 5 agents = clean context + faster execution
- Each agent reports pass/fail summary, not every test detail
Batching strategy:
- Group tests by feature area or MCP server
- 10-20 tests per agent is ideal
- Too few = overhead of spawning not worth it
- Too many = agent context fills up
MCP Testing
For MCP servers, the testing agent inherits configured MCPs:
claude mcp add --transport http gmail https://gmail.mcp.example.com/mcp
"Run tests for gmail MCP"
Example MCP test spec:
name: Gmail Search Tests
defaults:
tool: gmail_messages
tests:
- name: search_from_person
params: { action: search, searchQuery: "from John" }
expect: { contains: "from:john" }
- name: search_with_date
params: { action: search, searchQuery: "emails from January 2026" }
expect: { matches: "after:2026" }
API Testing
For REST APIs, use Bash tool:
name: API Tests
defaults:
timeout: 5000
tests:
- name: health_check
