DatattentionWhen LiteLLM is worth the proxy
Use a proxy when you have more than one upstream. Skip it when you have one local daemon.
LiteLLM is a proxy that presents one OpenAI-shaped API and forwards calls to many providers. That is worth it when an application should not know whether a request went to a local model or to a hosted one, or when you want one place for keys, timeouts, and fallbacks.
It is not worth it in front of a single Ollama process on the same machine. The extra hop adds a config file and a second thing to patch. Call the daemon directly until a second upstream exists.
If you do run it, bind it like any other internal service. The default port people remember is 4000. Treat that as an implementation detail, not as an invitation to leave 4000 open on a cloud host. Put the proxy on a private interface and publish only the hostname you intend.
Spend the configuration budget on timeouts, max tokens, and a fallback you have tested. A fallback that calls a provider you did not budget for is a surprise invoice. A fallback that calls a model you did not evaluate is a surprise answer.
Log the model requested, the model actually used, the latency, and the status. That is the record you need when someone says the assistant got worse on a Tuesday.
On this host
The LiteLLM-style proxy on this host is port 4000. It speaks the same chat route as the other OpenAI-compatible listeners, and it does not require a key. GET /v1/models lists gpt-4o-mini, claude-opus-5-5, claude-fable-5-1, and gpt-6-astra. Use it when you want one OpenAI-shaped URL that is not the Ollama port.
curl http://datattention.com:4000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello"}]}'The full list of ports and paths is on the endpoints page.