Private LLM deployment: when you need it, how to choose, what to prepare

With a private LLM deployment, ask first whether the data can leave the company, then think about models and compute. A private deployment runs the model, knowledge base and agents in an environment the company controls (its own data centre, a dedicated cloud or an isolated cloud environment), so business data isn’t sent to a public model service. It solves data control at the cost of higher build and operating effort.

When you need it

Any one of these is worth a serious look:

• Data can’t leave the company. Contracts, tax data, customer records, unpublished copyrighted content, or a regulation or customer contract that requires data to stay internal.
• Agents need internal systems. If an agent reads or writes ERP, finance or content systems on the internal network, permissions are easier to control inside.
• Call volume is large and steady. At sustained high volume, self-hosting can be more predictable than paying per call; at low or spiky volume it’s usually the reverse.

If staff mainly use AI for writing and research with non-sensitive data, a hosted API or an enterprise account is simpler.

Private deployment vs hosted API

	Private deployment	Hosted model API
Where data goes	Stays in an environment you control	Sent to the provider, kept and used under its terms
Time to start	Compute, deployment and tuning first	Call it as soon as the account is open
Model choice	Mainly open or licensed models that run locally	The latest commercial models directly
Cost	Up-front hardware or dedicated resources, then operations	Pay per call
Maintenance	You or your delivery partner handle upgrades, monitoring and security	The provider

Hybrids are common: sensitive data runs privately, non-sensitive tasks call a hosted service.

Five decisions before deploying

1. Task and acceptance. Define what it must do, build an evaluation set from real tasks and agree an acceptable error margin. Compare models on that set, not on parameter count.
2. Model and compute. Size the model to task difficulty and concurrency, then work out the GPUs and memory needed. Start with a smaller model and move up only if evaluation falls short.
3. Knowledge and data. Which material goes into the knowledge base, who keeps it current and which teams can see what. Enforce permissions at retrieval, not just in prompts.
4. Security and permissions. Private doesn’t mean secure by default: internal over-access, agents acting beyond their permissions, and model and dependency provenance each need design and audit.
5. Operations and handover. Logs, monitoring, model upgrades, human takeover when it fails, and who maintains it after the project ends.

Private deployments Moments has delivered

• Guomai Culture: 400 AI reading-companion agents on a knowledge graph covering 10,000 books, privately deployed so copyrighted content isn’t sent to a public model service.
• Think-Bridge: VatClaw, a VAT filing automation agent, privately deployed so tax data is processed in an environment the client controls.

Both cases describe delivery scope, not measured results. See AI agent development or the case studies.