LiteLLM

About

LiteLLM is a unified interface to access multiple LLMs (100+ LLMs). It provides consistent output, retry/fallback logic across multiple deployments, and tools for tracking spend and setting budgets per project. It can be used through a proxy server (LLM Gateway) or a Python SDK. The proxy server offers a central service to access multiple LLMs, track LLM usage and setup guardrails, and customize logging, guardrails, and caching per project. The Python SDK allows developers to use LiteLLM in their python code, providing retry/fallback logic and consistent output.

Features & Capabilities

• —Provides a consistent output format for various LLMs, with text responses always available at ['choices'][0]['message']['content']
• —Offers retry/fallback logic across multiple LLM deployments
• —Enables tracking of LLM usage and setting of budgets per project
• —Provides a unified interface for accessing multiple LLMs (100+) through a proxy server or Python SDK

Industry Focus