AIHostBox
Client area Deploy an app
All 57 applications

Categories

AI assistants & chatLLM runtimesAgents & RAGGateways & LLM opsVector & dataAutomationSpeech, image & videoSearch & web agentsSupport & CXML platformMCP servers

Solutions

Private team chatUnlimited automationCompany knowledge AIAI gateway & observability

More

PricingPrivate LLM hosting LicensingSecurity SupportClient area

Private LLM hosting

Run open-weight models behind your own API, on hardware nobody else shares.

from $13.99/mo CPU inferenceno token meter
In short

Private LLM hosting means running an open-weight model — Llama, Qwen, Mistral, Hermes — on a dedicated instance that exposes an OpenAI-compatible API only to you. Nothing is metered and no prompt leaves the box. On CPU plans, expect roughly 5-15 tokens per second from a quantized 7-8B model: comfortable for chat, retrieval and automation, not for realtime products.

There are two honest reasons to run your own model. The first is data: some prompts should not leave your infrastructure, and the cleanest way to guarantee that is for there to be no outbound call at all. The second is cost shape — a flat monthly instance instead of a meter that rewards you for using it less.

What you should not expect is frontier-model quality or frontier-model speed. A quantized 8B model is a capable assistant for summarising, extracting, classifying and drafting; it is not Claude or GPT, and on CPU it answers at reading speed rather than instantly. The teams who are happiest with this stack use a local model for volume work and a commercial API for the hard questions — which is exactly what an LiteLLM gateway is for.

What the plans actually run

PlanModel sizeRealistic throughput
8 GB / 4 vCPU7-8B quantized (Q4)~5-15 tokens/sec
12 GB / 6 vCPU13B quantized (Q4)Slower; batch and background work
GPU70B and realtimeNot offered — a specialist GPU cloud is the right buy

Throughput depends on the model, the quantization and the context length. These are the numbers we expect to hold, not best-case figures.

Runtimes we host

Ollama logo

Ollama

Your private LLM server — pull a model, get an API, keep every token on your own box.

from $16.99/moDetails →Hermes logo

Hermes

Nous Research’s Hermes models, served privately behind your own API.

from $16.99/moDetails →LocalAI logo

LocalAI

A drop-in OpenAI replacement for text, images, audio and embeddings.

from $16.99/moDetails →KoboldCpp logo

KoboldCpp

A single-binary GGUF server with its own UI and a KoboldAI API.

from $13.99/moDetails →llama.cpp logo

llama.cpp

The reference GGUF inference server, with nothing between it and the CPU.

from $13.99/moDetails →Text Generation WebUI logo

Text Generation WebUI

Oobabooga’s all-in-one interface for running and testing local models.

from $16.99/moDetails →vLLM logo

vLLM

High-throughput model serving — GPU territory, listed for completeness.

on requestDetails →
How it works

Three steps, no Docker knowledge required

Pick the app and the size

Choose memory and storage from the plan table. Every plan is a single-tenant instance with its own volume, its own configuration and its own admin account.

We deploy and harden it

TLS on your domain or ours, firewall, a version pinned to a reviewed release, nightly off-box backups and isolated secrets.

You log in and build

The admin account is yours. Add your API keys, invite your team, export your data whenever you want. Patching stays with us.

Questions

Straight answers

Can I run a large language model without a GPU?
Yes, within limits that are worth stating plainly. Quantized 7-8B models run at roughly 5-15 tokens per second on dedicated CPU cores — genuinely usable for chat, retrieval and background automation. 13B works on a 12 GB plan for non-realtime jobs. A 70B model or realtime latency needs GPU hardware, and we do not pretend otherwise.
Which model should I start with?
For general chat and RAG, a current 7-8B instruction model in a Q4 quantization is the sweet spot on CPU. If your workload is classification or extraction rather than conversation, smaller models often perform just as well and leave headroom.
How much RAM does a model need?
Roughly: model file size plus context. A 7-8B Q4 build is around 4-5 GB, so an 8 GB plan holds it with room for the context window; 13B Q4 wants 12 GB. We size plans against real model files rather than a marketing table.
Is anything sent to a third party?
No. The whole point of this stack is that the prompt, the model and the output stay on your instance. There is no provider call and no token meter.
Do you offer GPU plans?
Not yet. GPU capacity priced honestly is several hundred dollars a month, and we would rather send you to a specialist than sell an expensive compromise. If your workload needs one, tell us and we will say so.
Is this a shared account or my own instance?
Your own. Every plan is a single-tenant instance with its own storage, its own configuration and its own admin account. Nobody else’s workload runs inside it, and nothing you store is pooled with other customers.
Do I have to know Docker?
No. We install the application, put it behind TLS on your domain or a subdomain of ours, and hand you the login. If you do want shell-level access to your data we provide SFTP; you are never required to use it.
What is included in the monthly price?
The instance, the storage, the bandwidth, nightly off-box backups, monitoring, security patching and human support. Model tokens are not included — you bring your own provider key, so there is no markup on usage.
Can I move to a bigger plan later?
Yes, resizing is an in-place upgrade and your data stays where it is. If you outgrow the catalogue entirely we will help you export everything.

Run a model nobody else can read

Pick a runtime and a size. Your endpoint, your models, no metering.

Browse the catalogue Talk to a human