llm▌

Intelligent model routing, budget tracking, and retry logic to optimize LLM API costs without sacrificing quality. \n \n Routes requests to cheaper models (Haiku) for simple tasks and expensive models (Sonnet, Opus) only when complexity thresholds are met, reducing spend by 3–19x on routine work \n Tracks cumulative API costs with immutable dataclasses, enforces budget limits, and fails early to prevent overspend \n Implements narrow retry logic that retries only on transient errors (network, ra

Systematic evaluation of LLM applications using automated metrics, human feedback, and statistical testing. \n \n Covers three evaluation approaches: automated metrics (BLEU, ROUGE, BERTScore, accuracy, precision/recall), human evaluation across dimensions like accuracy and coherence, and LLM-as-Judge for pointwise, pairwise, and reference-based scoring \n Includes implementations for text generation, classification, and retrieval (RAG) evaluation with ready-to-use metric functions and custom me