Do I need all five layers?

No. Most teams run happily on an interface plus a provider key for months. Add a layer when something breaks or costs more than it should, not on principle.

Can these run on one instance?

Technically yes, and we do not recommend it. One application per instance keeps updates and failures isolated, which is most of what you are paying for. Stacks that genuinely belong together — an app with its own database and cache — we size as one deployment.

What order should I add them in?

Interface, gateway, then whichever of local model or retrieval your actual usage demands. Observability once more than one thing is in production.

Is this cheaper than ChatGPT Team?

For ten people, usually yes, and the gap widens with headcount because nothing here is priced per seat. For two people it is probably not, and we will say so.

The Self-Hosted AI Stack — What to Run Together

Self-hosted AI is usually assembled in the wrong order: a vector database before there is anything to retrieve, five tools before one is in daily use. Here is a stack that holds together, and the sequence that gets you there without waste.

The five layers

Nearly every working self-hosted setup is the same five pieces, whatever the logos: an interface people actually open, a gateway that holds the keys and the routing, one or more models, a retrieval layer over your own material, and observability so you know what happened and what it cost.

You do not need all five on day one. You do need to know which layer a new tool belongs to, because most stack confusion comes from adding a second thing to a layer that already had one.

Start with the interface

Open WebUI ($5.99/mo) is the usual first instance and often the only one a small team ever needs. Accounts, history, document upload with built-in retrieval, model presets. Point it at a provider key and people are using it the same afternoon.

Resist the urge to build a RAG pipeline first. Watch what people actually ask for a fortnight — the answer reshapes everything downstream.

Add the gateway before the second application

The moment a second thing needs model access, put LiteLLM ($5.99/mo) in front. It gives you one OpenAI-compatible endpoint, provider keys held in one place, virtual keys per team or app with budgets attached, fallbacks when a provider degrades, and a spend log.

Doing this early is much cheaper than retrofitting it. Applications only ever know the gateway address; what sits behind it becomes an operational decision rather than a code change.

Add a local model when the bill or the data says so

Two triggers: cost — high-volume classification and extraction dominating the invoice — or sensitivity, where certain prompts should not leave your network at all. Ollama ($16.99/mo) behind the gateway covers both, and the CPU throughput figures tell you which jobs fit.

Routing is by model name, so migrating a workload from a frontier model to a local one is a configuration change, not a rewrite.

Retrieval, when there is something worth retrieving

Two routes. AnythingLLM ($18.99/mo) is a complete workspace — upload documents, get cited answers, embedded vector store included — and is the right answer when the job is “ask questions about our documents”.

A dedicated vector store such as Qdrant ($5.99/mo) is the right answer when several applications share an index, when you want to query it directly, or when the corpus outgrows an embedded store. Our vector database comparison covers choosing between them — and makes the case for pgvector when you already run Postgres.

Observability last, and sooner than feels necessary

Langfuse ($11.99/mo) records traces, latency and cost per call, with prompt versioning and evaluation on top. It feels premature until the first time an agent gives a wrong answer and nobody can reconstruct why.

If everything already routes through LiteLLM, pointing that at Langfuse gives you coverage without instrumenting each application.

What a complete small-team stack costs

Interface, gateway, local model, retrieval and observability — five single-tenant instances — comes to roughly $60 a month at our prices, with no per-seat charge and no token markup. Model spend goes to your provider at list price on top, and drops as more of the volume moves local.

Compared against per-seat AI subscriptions for a team of ten, this is normally the cheaper side by a wide margin. Compared against one person’s afternoon per month of maintenance, it is roughly a wash — which is the honest case for having someone else run it.

A self-hosted AI stack that actually holds together