Systematic evaluation framework for AI products using practitioner-driven methodologies.
Works with
Guides users through understanding what \"good\" looks like, designing rubrics and test cases, and implementing scoring criteria aligned with actual user needs
Emphasizes manual review and error analysis as prerequisites to building meaningful evals, with structured workflows for clustering failure patterns
Flags common pitfalls including vague criteria, LLM-as-judge without validation, and Liker
AI-first code editor with Composer
Before installing skills in Cursor, ensure your development environment meets these requirements:
node --versionai-evalsExecute the skills CLI command in your project's root directory to begin installation:
Fetches ai-evals from refoundai/lenny-skills and configures it for Cursor.
The CLI shows a list of agents. Use arrow keys and space to select Cursor:
Confirm successful installation by checking the skill directory location:
Restart Cursor to activate ai-evals. Access via /ai-evals in your agent's command palette.
We perform automated surface-level scans (Gen AI Scanner, Socket, Snyk) during installation. These checks detect common vulnerabilities but do not guarantee complete security. Always review skill source code and verify the publisher's reputation before production use.
Skills execute code in your environment. Always review source, verify the publisher, and test in isolation before production.
Submit your Claude Code skill and start earning
Automate repetitive workflows and reduce manual effort
Example
Generate reports, summarize documents, draft communications
Save 3-5 hours per week on routine tasks
Learn new skills, understand complex topics, get expert guidance
Example
Explain concepts, provide examples, suggest learning resources
Accelerate learning and skill development by 2x
Enhance output quality through reviews, suggestions, and refinements
Example
Review drafts, suggest improvements, catch errors
Improve work quality by 30-40% with less effort
0
total installs
0
this week
616
GitHub stars
0
upvotes
Run in your terminal
0
installs
0
this week
616
stars
Help the user create systematic evaluations for AI products using insights from AI practitioners.
When the user asks for help with AI evals:
Brendan Foody: "If the model is the product, then the eval is the product requirement document." Evals define what success looks like in AI products—they're not optional quality checks, they're core specifications.
Hamel Husain & Shreya Shankar: "Both the chief product officers of Anthropic and OpenAI shared that evals are becoming the most important new skill for product builders." This isn't just for ML engineers—product people need to master this.
Building good evals involves error analysis, open coding (writing down what's wrong), clustering failure patterns, and creating rubrics. It's a systematic process, not a one-time test.
For all 2 insights from 2 guests, see references/guest-insights.md
Prerequisites
Time Estimate
15-45 minutes depending on use case complexity
Steps
Common Pitfalls
✓ Do
✗ Don't
💡 Pro Tips
✓ Use when
Use when skill capabilities match your task, clear ROI on time saved, and you can validate outputs. Best for repetitive tasks, learning, and quality improvement.
✗ Avoid when
Avoid when task requires deep expertise you can't validate, involves sensitive decisions, or when learning process is more valuable than speed of completion.
davila7/claude-code-templates
intellectronica/agent-skills
am-will/codex-skills
sickn33/antigravity-awesome-skills
myzy-ai/dokie-ai-ppt
sickn33/antigravity-awesome-skills
Keeps context tight: ai-evals is the kind of skill you can hand to a new teammate without a long onboarding doc.
We added ai-evals from the explainx registry; install was straightforward and the SKILL.md answered most questions upfront.
ai-evals is among the better-maintained entries we tried; worth keeping pinned for repeat workflows.
ai-evals fits our agent workflows well — practical, well scoped, and easy to wire into existing repos.
ai-evals has been reliable in day-to-day use. Documentation quality is above average for community skills.
Solid pick for teams standardizing on skills: ai-evals is focused, and the summary matches what you get after install.
Registry listing for ai-evals matched our evaluation — installs cleanly and behaves as described in the markdown.
Keeps context tight: ai-evals is the kind of skill you can hand to a new teammate without a long onboarding doc.
Useful defaults in ai-evals — fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
ai-evals has been reliable in day-to-day use. Documentation quality is above average for community skills.
showing 1-10 of 63