The easiest way to run an open model on your own machine: one command downloads and starts it, plus a local REST API and libraries for Python and JavaScript.
All repositories
Open models & inferenceBerriAI
litellm
A single gateway to 100+ LLM providers in OpenAI format: a Python SDK and a proxy server with cost tracking, load balancing and logging. Swap models without changing code.
Who it is for
For developers and teams working with several model providers.
How to start
- For the SDK:
uv add litellm. - For the proxy server:
uv tool install 'litellm[proxy]', thenlitellm --model gpt-4o. - Call the proxy through the OpenAI client with
base_url="http://0.0.0.0:4000".
Steps are taken from the README. Check the current version in the repository before running them.
Stars over the last 30 days
+2,224Sep 6 — Oct 6
Author's description
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
More in «Open models & inference»
The core library for working with models: one codebase for text, vision, audio and multimodal tasks, for both inference and training. Most new open models ship through it.
A self-hosted ChatGPT-style interface that connects to Ollama and any OpenAI-compatible API. Installs with a single Docker command and runs on your own server.
A C/C++ engine that runs language models on ordinary hardware: laptops, phones, GPU-less servers. Much of local AI, Ollama included, is built on it.
Repository of the large open DeepSeek-V3 mixture-of-experts model: description, benchmark results and run instructions. It showed an open model can stand next to closed ones.
The foundation under most modern neural networks: GPU-accelerated tensors and automatic differentiation, wrapped in ordinary Python. Transformers, Llama and nearly every open model are written on it.