All repositories

litellm

A single gateway to 100+ LLM providers in OpenAI format: a Python SDK and a proxy server with cost tracking, load balancing and logging. Swap models without changing code.

Who it is for

For developers and teams working with several model providers.

How to start

  1. For the SDK: uv add litellm.
  2. For the proxy server: uv tool install 'litellm[proxy]', then litellm --model gpt-4o.
  3. Call the proxy through the OpenAI client with base_url="http://0.0.0.0:4000".

Steps are taken from the README. Check the current version in the repository before running them.

Stars over the last 30 days

+1,715Oct 23 — Nov 20
61,85963,574

Author's description

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

ollama

ollama

The easiest way to run an open model on your own machine: one command downloads and starts it, plus a local REST API and libraries for Python and JavaScript.

188Kstars+3.3K in 29 dGo

huggingface

transformers

The core library for working with models: one codebase for text, vision, audio and multimodal tasks, for both inference and training. Most new open models ship through it.

169Kstars+1.3K in 29 dPython

open-webui

open-webui

A self-hosted ChatGPT-style interface that connects to Ollama and any OpenAI-compatible API. Installs with a single Docker command and runs on your own server.

159Kstars+2.5K in 28 dSvelte

ggml-org

llama.cpp

Growing fastest

A C/C++ engine that runs language models on ordinary hardware: laptops, phones, GPU-less servers. Much of local AI, Ollama included, is built on it.

136Kstars+3.7K in 29 dC++

pytorch

pytorch

The foundation under most modern neural networks: GPU-accelerated tensors and automatic differentiation, wrapped in ordinary Python. Transformers, Llama and nearly every open model are written on it.

106Kstars+674 in 30 dPython

deepseek-ai

DeepSeek-V3

Repository of the large open DeepSeek-V3 mixture-of-experts model: description, benchmark results and run instructions. It showed an open model can stand next to closed ones.

105Kstars+303 in 29 dPython