Sovereign LLM Workbench · CPU-native inference · informational procurement guide
Size customer-controlled hosts for local-sovereign AI (no public cloud LLM APIs): workbench + approved small models, retrieval over internal documents, and streaming chat. Figures are illustrative, not a quote or SLA.
| Tier | Users | Architecture | Hardware band |
|---|---|---|---|
| T0 | 0–5 | Single host | Laptop or small CPU server, 16 GB+ RAM |
| T1 | 6–10 | Split workbench / inference | Dedicated CPU server, 32 GB+ RAM |
| T2 | 10–15 | Split | CPU server with faster storage |
| T3 | 15–25 | Split | CPU cluster or larger memory; GPU optional |
| T4 | 25–50 | Pooled inference | Multiple CPU hosts; GPU still optional |
Capacity depends on model size, concurrent sessions, and operator policy. Not a cloud-vendor SKU list and not a price commitment.
Start with CPU hosts that meet the T3 memory band. Add GPU only if you later enable optional in-jurisdiction training — not for running the workbench.
| Do | Don't |
|---|---|
| Keep inference private to the workbench network | Expose inference ports to the public internet |
| Use TLS and authenticated APIs | Oversubscribe a single small host for many concurrent users |
| Approved catalog models only | Send regulated data to public cloud chat APIs |