vLLM hosting

High-throughput model serving — GPU territory, listed for completeness.

LLM runtimesApache-2.0on request

In short

vLLM is in our catalogue but not yet orderable: it needs GPU capacity, and we would rather list it honestly than sell a CPU plan that disappoints. Tell us what you need and we will tell you where it stands.

Paged attentionContinuous batchingOpenAI-compatibleGPU required

vLLM is an inference and serving engine for large language models that began at UC Berkeley’s Sky Computing Lab and is now hosted by the PyTorch Foundation, with a large outside contributor base. Its core idea, PagedAttention, stores the attention key-value cache in fixed-size blocks, much as an operating system pages memory, so far more concurrent sequences fit on one GPU. Combined with continuous batching, that lets a single card serve many users at once through the OpenAI-compatible server started with vllm serve.

That design only pays off on accelerator hardware. The official images target NVIDIA GPUs of compute capability 7.5 or newer (T4, L4, A100, H100 and later), AMD Instinct and Radeon cards through ROCm, and Intel data-centre GPUs. Production serving means keeping model weights resident in GPU memory around the clock and batching requests from many clients, which is server work by definition. For CPU-only hosting, the Ollama and llama.cpp plans we already sell are the better fit.

What vLLM needs

Not deployed yet: the planned stack is the official vllm/vllm-openai container on a GPU host, run with --ipc=host, the Hugging Face cache on a persistent volume and the OpenAI-compatible server on port 8000 behind TLS.

What we configure for you

HF_TOKEN for gated model downloads from Hugging Face
--api-key so the endpoint rejects unauthenticated requests
--max-model-len to cap context at what the GPU’s KV cache can hold
--gpu-memory-utilization (default 0.9) adjusted where headroom is needed
--tensor-parallel-size when a model spans more than one GPU
A persistent ~/.cache/huggingface volume so restarts do not re-download weights

Which plan fits

There are no vLLM tiers today. A realistic entry point would be a single 24 GB GPU for 7-8B models at 16-bit, 48 GB or more for 13-34B models or long contexts, and multi-GPU nodes for 70B. We will publish tiers against those classes of card rather than approximations.

The honest limits

vLLM is GPU software and we do not operate GPU capacity yet, so nothing on this page can be ordered. For CPU serving of quantized models today, Ollama or llama.cpp is the honest recommendation.

What people run vLLM for

Serving an open model to many users

Continuous batching keeps throughput high when dozens of requests arrive at once.

Multi-LoRA serving

Load several fine-tuned LoRA adapters over one base model and choose one per request.

Structured output at scale

Constrain generation to a JSON schema for extraction pipelines that must parse every reply.

Offline batch inference

Run large datasets through the Python API instead of the HTTP server.

Licence and version policy

Item	Position
Upstream licence	Apache-2.0
Sold as managed hosting	No
Tenancy	Single tenant — one customer per instance
Patching	Critical advisories within 24 hours, high severity within 72

Our full position on licences, including the applications we decline to sell, is on the licensing page.

Reading on vLLM

ComparisonOllama vs vLLM

Works well with vLLM

More LLM runtimes we host

Questions

vLLM hosting: your questions

When will vLLM hosting be available?

When we can offer GPU capacity at a price we can hold. Until then this page is informational and there is no order button. If you need an open model served today, our Ollama and llama.cpp plans run quantized 7-13B models on CPU behind an OpenAI-compatible API, so client code written for vLLM carries over.

Can vLLM run on CPU at all?

The project does publish CPU builds, but its advantages, the paged KV cache and continuous batching across many sequences, exist to keep a GPU busy. On CPU it gives up that edge and is harder to size than Ollama or llama.cpp, which are built for quantized GGUF models on ordinary cores.

What GPU would an 8B model need?

At 16-bit precision an 8B model’s weights alone take about 16 GB, and vLLM by default claims 90% of GPU memory (--gpu-memory-utilization 0.9) for weights plus KV cache. A 24 GB card such as an L4 is the practical floor for 8B; 70B needs several 80 GB GPUs with tensor parallelism, or a quantized checkpoint.

Is vLLM’s API compatible with OpenAI clients?

Yes. vllm serve exposes /v1/chat/completions, /v1/completions and, for embedding models, /v1/embeddings on port 8000, and the --api-key flag adds bearer-token checks. Existing OpenAI SDK code usually needs only a new base URL and model name.

What licence is vLLM under?

Apache-2.0, which permits commercial and hosted use. Model weights carry their own licences, so a Llama, Qwen or Mistral checkpoint served through vLLM is still bound by its publisher’s terms, including gated downloads that need a Hugging Face token (HF_TOKEN).

Tell us what you need

If this is the app you need, say so — demand is how things move up the list.

Browse the catalogue Talk to a human

vLLM hosting

What vLLM needs

What we configure for you

Which plan fits

The honest limits

What people run vLLM for

Serving an open model to many users

Multi-LoRA serving

Structured output at scale

Offline batch inference

Licence and version policy

Reading on vLLM

Works well with vLLM

LiteLLM

Open WebUI

Ollama

Langfuse

More LLM runtimes we host

Hermes

LocalAI

KoboldCpp

llama.cpp

vLLM hosting: your questions

Tell us what you need