Productivity
langsmith-evaluator▌
langchain-ai/langsmith-skills · updated Apr 8, 2026
$npx skills add https://github.com/langchain-ai/langsmith-skills --skill langsmith-evaluator
summary
Build evaluation pipelines for LangSmith with LLM-as-Judge and custom code evaluators.
- ›Three core components: creating evaluators (LLM-as-Judge or custom code), defining run functions to capture agent outputs and trajectories, and running evaluations locally or auto-running via uploaded evaluators
- ›Supports both offline evaluators (comparing run outputs to dataset examples) and online evaluators (real-time quality checks on production runs)
- ›Requires LangSmith API key and project confi
skill.md
langsmith-evaluator
No content available.