llama.cpp hosting

The reference GGUF inference server, with nothing between it and the CPU.

LLM runtimesMITfrom $13.99/mo

In short

llama.cpp hosting at AIHostBox means a dedicated, single-tenant llama.cpp instance on our infrastructure: we install it, pin it to a reviewed release, put it behind TLS on your domain, back it up nightly and keep it patched. Plans start at $13.99 per month and the admin account, the data and any API keys are yours.

GGUFOpenAI-compatibleGrammarsLowest overhead

llama.cpp is the C/C++ inference engine maintained by the ggml-org project, and the base that Ollama, KoboldCpp, LM Studio and many others build on. Its bundled llama-server exposes an OpenAI-compatible chat and completions API, an embeddings endpoint, a small built-in web UI and native endpoints for health, slots and Prometheus metrics. Models are GGUF files, downloaded directly or pulled from Hugging Face. The same binary also serves reranking, tokenisation and slot inspection, so one process can back a chat front-end and a RAG pipeline at once.

Running the server itself, rather than a wrapper, means every flag is yours: thread count, context size, parallel slots, KV-cache behaviour, chat templates and JSON-schema constraints. That control matters most on CPU, where the difference between a default and a tuned setup is noticeable. A server keeps the model loaded and reachable for every application that needs it, instead of tying it to a laptop that sleeps. Other applications on your account can call it by URL with the API key, just as they would call a commercial provider.

llama.cpp hosting plans

Starter7-8B GGUF (Q4)

8 GB RAM · 4 vCPU · NVMe

$13.99/mo

Order

Plus13B GGUF (Q4)

12 GB RAM · 6 vCPU · NVMe

$24.99/mo

Order

Every plan is a dedicated single-tenant instance with NVMe storage, nightly off-box backups, TLS and managed patching. Quarterly, semi-annual and annual terms are discounted at checkout. Model tokens are not included — you bring your own key.

How we run llama.cpp

The official ghcr.io/ggml-org/llama.cpp:server image running llama-server with one GGUF model on a persistent models volume; there is no database.

What we configure for you

LLAMA_ARG_THREADS matched to the plan’s vCPUs
LLAMA_ARG_CTX_SIZE and LLAMA_ARG_N_PARALLEL sized so context and slots fit in RAM
LLAMA_ARG_API_KEY set, with TLS on your hostname
Model of your choice downloaded to the models volume (Hugging Face GGUF repos via -hf)
LLAMA_ARG_ENDPOINT_METRICS for the Prometheus /metrics endpoint, if you want it

Which plan fits

Starter (8 GB, 4 vCPU) holds one 7-8B GGUF model at Q4 with a reasonable context. Plus (12 GB, 6 vCPU) moves you to 13B at Q4, or leaves room for a longer context and more parallel slots on a smaller model.

The honest limits

These are CPU plans, so throughput is 5-15 tokens/s on 7-8B Q4, and GPU offload flags such as n-gpu-layers do nothing here. You are responsible for choosing sensible flags; there is no model manager.

What people run llama.cpp for

Squeeze the most from CPU

Tune threads and batch size for your specific model.

Structured output

Grammar constraints force valid JSON without retry loops.

Private embeddings endpoint

Serve a GGUF embedding model for a RAG app on the same network.

Licence and version policy

Item	Position
Upstream licence	MIT
Sold as managed hosting	Yes
Tenancy	Single tenant — one customer per instance
Patching	Critical advisories within 24 hours, high severity within 72

Our full position on licences, including the applications we decline to sell, is on the licensing page.

Reading on llama.cpp

ComparisonOllama vs vLLM ComparisonOllama vs LM Studio Guide · Private LLMRunning an LLM on CPU: what actually works

Works well with llama.cpp

More LLM runtimes we host

Questions

llama.cpp hosting: your questions

Which models can llama.cpp run?

Any model converted to GGUF, which covers the Llama, Qwen, Mistral, Gemma, Phi and DeepSeek families among many others. On our CPU plans that means 7-8B models at Q4 on Starter and 13B at Q4 on Plus. Larger quantisations or bigger models will not fit in memory.

How fast is it on CPU?

Expect roughly 5-15 tokens per second on a 7-8B Q4 model, depending on the model, context length and how many requests run in parallel. That is comfortable for a chat front-end or background jobs, not for high-concurrency public traffic.

Is the endpoint protected?

Yes. We start llama-server with an API key (the --api-key option, or LLAMA_ARG_API_KEY), so every request must carry it as a bearer token, exactly as an OpenAI client would send it. The endpoint is served over TLS on your own hostname.

How do I get structured JSON out of it?

llama-server accepts a GBNF grammar or a JSON schema with the request and constrains sampling so the output must match. That removes the parse-and-retry loop you often need with free-form output from small models.

How is llama.cpp different from your Ollama plan?

Ollama wraps llama.cpp with a model library and automatic management; llama.cpp gives you the raw server and every tuning flag. Choose Ollama for convenience and model switching, llama.cpp when you want one model configured precisely and nothing in between.

Run llama.cpp without running a server

A dedicated llama.cpp instance, deployed and maintained by us. Month to month, cancel whenever.

Browse the catalogue Talk to a human

llama.cpp hosting

llama.cpp hosting plans

How we run llama.cpp

What we configure for you

Which plan fits

The honest limits

What people run llama.cpp for

Squeeze the most from CPU

Structured output

Private embeddings endpoint

Licence and version policy

Reading on llama.cpp

Works well with llama.cpp

Open WebUI

SillyTavern

LiteLLM

AnythingLLM

More LLM runtimes we host

Ollama

Hermes

LocalAI

KoboldCpp

llama.cpp hosting: your questions

Run llama.cpp without running a server