Jump to content

LiteLLM

Add links
From Wikipedia, the free encyclopedia


🚅 LiteLLM
DeveloperBerriAI
Release2023
Written inPython and TypeScript
TypeLLM gateway
LicenseMIT License
Websitewww.litellm.ai
Repositoryhttps://github.com/BerriAI/litellm

LiteLLM is an open-source large language model (LLM) gateway developed by BerriAI. Founded by Ishaan Jaffer and Krrish Dholakia[1]. It provides a unified OpenAI-compatible application programming interface (API) for accessing and routing requests to different model providers. LiteLLM can be deployed as a self-hosted proxy server or used as a Python library within applications.

Functionality

[edit]

LiteLLM provides a common interface for models and services from providers including OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Microsoft Azure, and Hugging Face. It translates requests and responses between provider APIs into a consistent OpenAI-compatible format, allowing applications to switch between models without separately implementing each provider's interface.[2][3]

The software also provides model routing, load balancing, fallbacks, authentication, rate limiting, usage monitoring, and cost controls. Its proxy server can act as a centralized API gateway through which multiple applications or users access AI models.[3]

LiteLLM also supports the Model Context Protocol (MCP), an open protocol developed by Anthropic for connecting AI applications to external tools and data sources.[4]

Security

[edit]

In March 2026, malicious versions of LiteLLM were distributed through the Python Package Index (PyPI) as part of a software supply-chain attack. TechCrunch reported that AI recruiting company Mercor was among the organizations affected by the compromise.[5]

Compatibility

[edit]

Supported models

[edit]

LiteLLM supports more than 100 large language models through a common OpenAI-compatible interface. Notable supported model families include:[3]

Compatible AI coding software

[edit]

LiteLLM can be used as an inference gateway for AI-assisted software development tools, routing their model requests through the LiteLLM proxy. Documented integrations include:[6]

See also

[edit]

References

[edit]
  1. ↑ "LiteLLM: Call every LLM API like it's OpenAI [100+ LLMs]". Y Combinator. Retrieved October 11, 2026.
  2. ↑ MSV, Janakiram (May 15, 2025). "LiteLLM: An open-source gateway for unified LLM access". InfoWorld. Retrieved September 9, 2026.
  3. 1 2 3 "LiteLLM - Getting Started". LiteLLM. BerriAI. Retrieved September 9, 2026.
  4. ↑ "Introducing the Model Context Protocol". Anthropic. November 25, 2024. Retrieved September 9, 2026.
  5. ↑ Singh, Jagmeet (March 31, 2026). "Mercor says it was hit by cyberattack tied to compromise of open source LiteLLM project". TechCrunch. Retrieved September 9, 2026.
  6. ↑ "LiteLLM Proxy CLI". GitHub. BerriAI. Retrieved September 10, 2026.
  7. ↑ "Configure LiteLLM Provider". Cline. Retrieved September 10, 2026.
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=LiteLLM&oldid=1379785410"