Private deployment sizing — executive summary

Sovereign LLM Workbench · CPU-native inference · informational procurement guide

Purpose

Size customer-controlled hosts for local-sovereign AI (no public cloud LLM APIs): workbench + approved small models, retrieval over internal documents, and streaming chat. Figures are illustrative, not a quote or SLA.

Experience targets (illustrative)

Tier summary (licensed users)

TierUsersArchitectureHardware band
T00–5Single hostLaptop or small CPU server, 16 GB+ RAM
T16–10Split workbench / inferenceDedicated CPU server, 32 GB+ RAM
T210–15SplitCPU server with faster storage
T315–25SplitCPU cluster or larger memory; GPU optional
T425–50Pooled inferenceMultiple CPU hosts; GPU still optional

Capacity depends on model size, concurrent sessions, and operator policy. Not a cloud-vendor SKU list and not a price commitment.

Recommendation — 25 users, interactive chat

Start with CPU hosts that meet the T3 memory band. Add GPU only if you later enable optional in-jurisdiction training — not for running the workbench.

Do / Don't

DoDon't
Keep inference private to the workbench networkExpose inference ports to the public internet
Use TLS and authenticated APIsOversubscribe a single small host for many concurrent users
Approved catalog models onlySend regulated data to public cloud chat APIs