Session Analyzer Skill
Post-hoc analysis tool for validating Claude Code session behavior against SKILL.md specifications.
Purpose
Analyze completed sessions to verify:
- Expected vs Actual Behavior - Did the skill follow SKILL.md workflow?
- Component Invocations - Were SubAgents, Hooks, and Tools called correctly?
- Artifacts - Were expected files created/deleted?
- Bug Detection - Any unexpected errors or deviations?
Input Requirements
| Parameter |
Required |
Description |
sessionId |
YES |
UUID of the session to analyze |
targetSkill |
YES |
Path to SKILL.md to validate against |
additionalRequirements |
NO |
Extra validation criteria |
Phase 1: Locate Session Files
Step 1.1: Find Session Files
Session files are located in ~/.claude/:
~/.claude/projects/-{encoded-cwd}/{sessionId}.jsonl
~/.claude/debug/{sessionId}.txt
~/.claude/projects/-{encoded-cwd}/agent-{agentId}.jsonl
Use script to locate files:
${baseDir}/scripts/find-session-files.sh {sessionId}
Step 1.2: Verify Files Exist
Check all required files exist before proceeding. If debug log is missing, analysis will be limited.
Phase 2: Parse Target SKILL.md
Step 2.1: Extract Expected Components
Read the target SKILL.md and identify:
From YAML Frontmatter:
hooks.PreToolUse - Expected PreToolUse hooks and matchershooks.PostToolUse - Expected PostToolUse hookshooks.Stop - Expected Stop hookshooks.SubagentStop - Expected SubagentStop hooksallowed-tools - Tools the skill is allowed to use
From Markdown Body:
- SubAgents mentioned (
Task(subagent_type="..."))
- Skills called (
Skill("..."))
- Artifacts created (
.dev-flow/drafts/, .dev-flow/plans/, etc.)
- Workflow steps and conditions
Step 2.2: Build Expected Behavior Checklist
Create checklist from SKILL.md analysis:
## Expected Behavior
### SubAgents
- [ ] Explore agent called (parallel, run_in_background)
- [ ] gap-analyzer called before plan generation
- [ ] reviewer called after plan creation
### Hooks
- [ ] PreToolUse[Edit|Write] triggers plan-guard.sh
- [ ] Stop hook validates reviewer approval
### Artifacts
- [ ] Draft file created at .dev-flow/drafts/{name}.md
- [ ] Plan file created at .dev-flow/plans/{name}.md
- [ ] Draft file deleted after OKAY
### Workflow
- [ ] Interview Mode before Plan Generation
- [ ] User explicit request triggers plan generation
- [ ] Reviewer REJECT causes revision loop
Phase 3: Analyze Debug Log
The debug log (~/.claude/debug/{sessionId}.txt) contains detailed execution traces.
Step 3.1: Extract SubAgent Calls
Search patterns:
SubagentStart with query: {agent-name}
SubagentStop with query: {agent-id}
Use script:
${baseDir}/scripts/extract-subagent-calls.sh {debug-log-path}
Step 3.2: Extract Hook Events
Search patterns:
Getting matching hook commands for {HookEvent} with query: {tool-name}
Matched {N} unique hooks for query "{query}"
Hooks: Processing prompt hook with prompt: {prompt}
Hooks: Prompt hook condition was met/not met
permissionDecision: allow/deny
Use script:
${baseDir}/scripts/extract-hook-events.sh {debug-log-path}
Step 3.3: Extract Tool Calls
Search patterns:
executePreToolHooks called for tool: {tool-name}
File {path} written atomically
Step 3.4: Extract Hook Results
For prompt-based hooks, find the model response:
Hooks: Model response: {
"ok": true/false,
"reason": "..."
}
Phase 4: Verify Artifacts
Step 4.1: Check File Creation
For each expected artifact:
- Search debug log for
FileHistory: Tracked file modification for {path}
- Search for
File {path} written atomically
- Verify current filesystem state
Step 4.2: Check File Deletion
For files that should be deleted:
- Search for
rm commands in Bash calls
- Verify file no longer exists on filesystem
Phase 5: Compare Expected vs Actual
Step 5.1: Build Comparison Table
|-----------|----------|--------|--------|
| Explore agent | 2 parallel calls | 2 calls at 09:39:26 | ✅ |
| gap-analyzer | Called before plan | Called at 09:43:08 | ✅ |
| reviewer | Called after plan | 2 calls (REJECT→OKAY) | ✅ |
| PreToolUse hook | Edit\|Write matcher | Triggered for Write | ✅ |
| Stop hook | Validates approval | Returned ok:true | ✅ |
| Draft file | Created then deleted | Created→Deleted | ✅ |
| Plan file | Created | Exists (10KB) | ✅ |
Step 5.2: Identify Deviations
Flag any mismatches:
- Missing component calls
- Wrong order of operations
- Hook failures
- Missing artifacts
- Unexpected errors
Phase 6: Generate Report
Report Template
# Session Analysis Report
## Session Info
- **Session ID**: {sessionId}
- **Target Skill**: {skillPath}
- **Analysis Date**: {date}
---
## 1. Expected Behavior (from SKILL.md)
[Summary of expected workflow]
---
## 2. Skill/SubAgent/Hook Verification
### SubAgents
|----------|----------|--------|------|--------|
| ... | ... | ... | ... | ✅/❌ |
### Hooks
|------|---------|-----------|--------|
| ... | ... | ... | ✅/❌ |
---
## 3. Artifacts Verification
|----------|------|----------------|--------------|
| ... | ... | ... | ✅/❌ |
---
## 4. Issues/Bugs
|----------|-------------|----------|
| ... | ... | ... |
---
## 5. Overall Result
**Verdict**: ✅ PASS / ❌ FAIL
**Summary**
