Agent Evaluation (AI Agent Evals)

Based on Anthropic's "Demystifying evals for AI agents"

When to use this skill

• Designing evaluation systems for AI agents
• Building benchmarks for coding, conversational, or research agents
• Creating graders (code-based, model-based, human)
• Implementing production monitoring for AI systems
• Setting up CI/CD pipelines with automated evals
• Debugging agent performance issues
• Measuring agent improvement over time

Core Concepts

Eval Evolution: Single-turn → Multi-turn → Agentic

7 Key Terms

Instructions

Step 1: Understand Grader Types

Code-based Graders (Recommended for Coding Agents)

• Pros: Fast, objective, reproducible
• Cons: Requires clear success criteria
• Best for: Coding agents, structured outputs

# Example: Code-based grader
def grade_task(outcome: dict) -> float:
    """Grade coding task by test passage."""
    tests_passed = outcome.get("tests_passed", 0)
    total_tests = outcome.get("total_tests", 1)
    return tests_passed / total_tests

# SWE-bench style grader
def grade_swe_bench(repo_path: str, test_spec: dict) -> bool:
    """Run tests and check if patch resolves issue."""
    result = subprocess.run(
        ["pytest", test_spec["test_file"]],
        cwd=repo_path,
        capture_output=True
    )
    return result.returncode == 0

Model-based Graders (LLM-as-Judge)

• Pros: Flexible, handles nuance
• Cons: Requires calibration, can be inconsistent
• Best for: Conversational agents, open-ended tasks

# Example: LLM Rubric for Customer Support Agent
rubric:
  dimensions:
    - name: empathy
      weight: 0.3
      scale: 1-5
      criteria: |
        5: Acknowledges emotions, uses warm language
        3: Polite but impersonal
        1: Cold or dismissive

    - name: resolution
      weight: 0.5
      scale: 1-5
      criteria: |
        5: Fully resolves issue
        3: Partial resolution
        1: No resolution

    - name: efficiency
      weight: 0.2
      scale: 1-5
      criteria: |
        5: Resolved in minimal turns
        3: Reasonable turns
        1: Excessive back-and-forth

Human Graders

• Pros: Highest accuracy, catches edge cases
• Cons: Expensive, slow, not scalable
• Best for: Final validation, ambiguous cases

Step 2: Choose Strategy by Agent Type

2.1 Coding Agents

Benchmarks:

• SWE-bench Verified: Real GitHub issues (40% → 80%+ achievable)
• Terminal-Bench: Complex terminal tasks
• Custom test suites with your codebase

Grading Strategy:

def grade_coding_agent(task: dict, outcome: dict) -> dict:
    return {
        "tests_passed": run_test_suite(outcome["code"]),
        "lint_score": run_linter(outcome["code"]),
        "builds": check_build(outcome["code"]),
        "matches_spec": compare_to_reference(task["spec"], outcome["code"])
    }

Key Metrics:

• Test passage rate
• Build success
• Lint/style compliance
• Diff size (smaller is better)

2.2 Conversational Agents

Benchmarks:

• τ2-Bench: Multi-domain conversation
• Custom domain-specific suites

Grading Strategy (Multi-dimensional):

success_criteria:
  - empathy_score: >= 4.0
  - resolution_rate: >= 0.9
  - avg_turns: <= 5
  - escalation_rate: <= 0.1

Key Metrics:

• Task resolution rate
• Customer satisfaction proxy
• Turn efficiency
• Escalation rate

2.3 Research Agents

Grading Dimensions:

1. Grounding: Claims backed by sources
2. Coverage: All aspects addressed
3. Source Quality: Authoritative sources used

def grade_research_agent(task: dict, outcome: dict) -> dict:
    return {
        "grounding": check_citations(outcome["report"]),
        "coverage": check_topic_coverage(task["topics"], outcome["report"]),
        "source_quality": score_sources(outcome["sources"]),
        "factual_accuracy": verify_claims(outcome["claims"])
    }

2.4 Computer Use Agents

Benchmarks:

• WebArena: Web navigation tasks
• OSWorld: Desktop environment tasks

Grading Strategy:

def grade_computer_use(task: dict, outcome: dict) -> dict:
    return {
        "ui_state": verify_ui_state(outcome["screenshot"]),
        "db_state": verify_database(task["expected_db_state"]),
        "file_state": verify_files(task["expected_files"]),
        "success": all_conditions_met(task, outcome)
    }

Step 3: Follow the 8-Step Roadmap

Step 0: Start Early (20-50 Tasks)

# Create initial eval suite structure
mkdir -p evals/{tasks,results,graders}

# Start with representative tasks
# - Common use cases (60%)
# - Edge cases (20%)
# - Failure modes (20%)

Step 1: Convert Manual Tests

# Transform existing QA tests into eval tasks
def convert_qa_to_eval(qa_case: dict) -> dict:
    return {
        "id": qa_case["id"],
        "prompt": qa_case["input"],
        "expected_outcome": qa_case["expected"],
        "grader": "code" if qa_case["has_tests"] else "model",
        "tags": qa_case.get("tags", [])
    }

Step 2: Ensure Clarity + Reference Solutions

# Good task definition
task:
  id: "api-design-001"
  prompt: |
    Design a REST API for user management with:
    - CRUD operations