GH Repository · BerriAI
litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
- stars
- 58,971
- 30-day movement
- starts with the next reading
- Related entries
- 60
- Connections
- 5
nodeagent-frameworkPythonrust-airustmcp-gatewayanthropicpythonllmpython/uvai-gatewayopenaiazure-openaillmopslangchainbedrockgatewayopenai-proxyllm-gatewaymakedockervertex-ailitellm
LiteLLM is an AI gateway with a Rust core and Python SDK that lets you call 100+ LLM APIs in OpenAI or native format. It includes cost tracking, guardrails, load balancing, and logging across providers like Bedrock, Azure, OpenAI, Anthropic, VertexAI, vLLM, and Nvidia NIM.
Reach for it when you want a single gateway layer to route, track, and control calls across many LLM providers.
Use it to
- Proxy LLM calls through one OpenAI-compatible gateway
- Track spend across multiple providers
- Apply guardrails to model requests
- Load-balance across providers or deployments
- Log LLM traffic centrally
For Teams running LLM apps across multiple providers
- Role
- agent-framework
- Language
- Python
- Licence
- custom (licence file present)
- Forks
- 11,526
- Open issues
- 1,733
- Last push
- 2026-09-17
- Latest release
- v1.43.5 · 2024-08-10
topicsllm-gatewayopenai-proxyllmopsload-balancingrustpython-sdk