langsmith-evaluator▌
langchain-ai/langsmith-skills · updated Apr 8, 2026
Build evaluation pipelines for LangSmith with LLM-as-Judge and custom code evaluators.
- ›Three core components: creating evaluators (LLM-as-Judge or custom code), defining run functions to capture agent outputs and trajectories, and running evaluations locally or auto-running via uploaded evaluators
- ›Supports both offline evaluators (comparing run outputs to dataset examples) and online evaluators (real-time quality checks on production runs)
- ›Requires LangSmith API key and project confi
langsmith-evaluator
No content available.
Discussion
Product Hunt–style comments (not star reviews)- No comments yet — start the thread.
Ratings
4.7★★★★★39 reviews- ★★★★★Benjamin Yang· Dec 28, 2024
Useful defaults in langsmith-evaluator — fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
- ★★★★★Ishan Ramirez· Dec 8, 2024
We added langsmith-evaluator from the explainx registry; install was straightforward and the SKILL.md answered most questions upfront.
- ★★★★★Daniel Jain· Nov 27, 2024
langsmith-evaluator fits our agent workflows well — practical, well scoped, and easy to wire into existing repos.
- ★★★★★Nikhil Bansal· Nov 19, 2024
langsmith-evaluator is among the better-maintained entries we tried; worth keeping pinned for repeat workflows.
- ★★★★★Aditi Shah· Oct 22, 2024
Solid pick for teams standardizing on skills: langsmith-evaluator is focused, and the summary matches what you get after install.
- ★★★★★Xiao Bhatia· Oct 18, 2024
langsmith-evaluator has been reliable in day-to-day use. Documentation quality is above average for community skills.
- ★★★★★Xiao Chawla· Oct 10, 2024
langsmith-evaluator reduced setup friction for our internal harness; good balance of opinion and flexibility.
- ★★★★★Piyush G· Sep 25, 2024
langsmith-evaluator fits our agent workflows well — practical, well scoped, and easy to wire into existing repos.
- ★★★★★Aditi Robinson· Sep 25, 2024
Useful defaults in langsmith-evaluator — fewer surprises than typical one-off scripts, and it plays nicely with `npx skills` flows.
- ★★★★★Aditi Tandon· Sep 1, 2024
langsmith-evaluator has been reliable in day-to-day use. Documentation quality is above average for community skills.
showing 1-10 of 39