One endpoint. Every model.

Nothing about your client code changes.

Every language already ships a mature OpenAI client that accepts a custom base_url. Point it at llmux and routing, budgets, caching and cost accounting happen underneath — no new SDK, no new wire format to learn.

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-team-a" \
  -H "Content-Type: application/json" \
  -d '{"model":"assistant",
       "messages":[{"role":"user","content":"Hello!"}]}'

{ "usage": {
    "prompt_tokens": 9, "completion_tokens": 12,
    "total_tokens": 21,
    "cost": { "input_cost":  0.0000014,
              "output_cost": 0.0000072,
              "total_cost":  0.0000086,
              "currency":    "USD" } } }

Fifteen languages. One engine behind all of them.

Fifteen language packages ship in this repo, and they are wrappers, not reimplementations — one routing engine, one sovereignty gate, one budget ledger, one price catalog, reached from all fifteen. What differs is only how your process reaches it, and the honest default is not the same in every language.

In your processDirect by default7

Go package import C C++ Rust Swift Deno Bun

A child process on loopbackSidecar by default7

Node.js Python Java Kotlin .NET / C# PHP Elixir

Each of the seven has a stated reason — a pre-forking host, a runtime whose signal handlers the library replaces, a thread that cannot be retired, or a platform with no library to load. Elixir has no direct mode at all, on purpose.

Your deployment decidesDepends1

Ruby

Unicorn, Passenger and clustered Puma fork, so they need the sidecar; single-mode Puma, Falcon, Sidekiq and CLI tools do not, and direct is fine there.

A first call in every language → The package index, and the measurement behind each recommendation →

The controls a fleet of applications actually needs — built in, not bolted on.

Every application gets its own virtual key rather than the master key. Caches never cross a key boundary, and every response already knows what it cost.

keysVirtual keys & budgets

Per-key USD budget_usd, rpm, and a model allow-list. Over budget is 402, over the RPM limit is 429. Spend persists to Postgres; tokens are sha256-hashed at rest.

cacheExact and semantic

LRU + TTL exact-match, or embedding-similarity semantic matching — in-memory or shared via Redis, scoped per virtual key so keys never see each other's completions.

pricingLive, offline-first

A built-in seed prices requests with zero network access; it auto-syncs from OpenRouter and LiteLLM when allowed. Merged catalog at GET /v1/catalog.json.

consoleEmbedded dashboard

Usage by model, key budgets, and the live catalog — served from the binary at /ui via go:embed. Plus Prometheus /metrics, structured logs, /health.

Build it, point a key at it, call it. No account to create.

Go 1.25+ and nothing else — the admin console is a hand-written file, so there's no Node toolchain anywhere in this repo. No database or cache is required for a single replica. Writing Go? Skip the binary — gateway.New builds the same dispatch path in-process, with no goroutines started and no sockets opened beyond an explicitly configured Postgres DSN.

# 1. Build the binary (embeds web/ui.html — no Node, no build step)
make build

# 2. Configure providers
export OPENAI_API_KEY=...
export ANTHROPIC_API_KEY=...

# 3. Run — gateway on :4000, dashboard at /ui
cp llmux.example.json llmux.json
./dist/llmux -config llmux.json

import (
    "github.com/vul-os/llmux/core/config"
    "github.com/vul-os/llmux/core/gateway"
    "github.com/vul-os/llmux/core/openai"
)

gw, err := gateway.New(config.Default())  // no goroutines, no sockets*
defer gw.Close()

res, err := gw.Chat(ctx, &openai.ChatCompletionRequest{
    Model:    "cheapest",
    Messages: []openai.Message{{Role: "user", Content: openai.Str("hi")}},
})

from openai import OpenAI

client = OpenAI(base_url="http://localhost:4000/v1", api_key="sk-team-a")

resp = client.chat.completions.create(
    model="cheapest",       # least-cost route from your config
    messages=[{"role": "user", "content": "hi"}],
)
print(resp.usage)
# usage.cost = {input_cost, output_cost, total_cost, currency}

{
  "keys": [
    { "key": "sk-team-a", "name": "team-a",
      "budget_usd": 25.0,          // 0 = unlimited
      "rpm": 60,
      "allowed_models": ["assistant", "cheapest"] }
  ]
}