EXECUTIVE BRIEFING
PSG EXECUTIVE BRIEFING · NO. 01
AI behind your firewall.
Private and offline large language models for businesses whose data cannot leave the building — why the open-weight generation changed the math, which deployment posture fits your risk profile, and a practical path from zero to a working private assistant.
PLATINUM STRATEGY GROUP · 2026
WWW.PSG-INC.COM
Some data should never ride someone else’s API.
For most companies, commercial AI services with enterprise terms are fine. But five classes of data change the calculus: client-confidential material under NDA, regulated personal data (health, financial, payroll), defense-adjacent or export-controlled information, live deal and litigation data, and anything your own contracts promise never leaves your control. The good news: open-weight models — the Llama, Mistral, and Qwen families — have closed enough of the quality gap that a model running on your own hardware now handles summarization, drafting, extraction, and internal Q&A credibly. Privacy no longer costs you the capability.
THREE DEPLOYMENT POSTURES
Fully offline, on-premise
Model weights, inference, and documents live on hardware you own; the system can run with the internet cable unplugged.
Fits: regulated data, air-gapped environments, contractual no-cloud obligations. Highest control, highest ops burden.
Private cloud (your VPC)
Open-weight models on GPU instances inside your own cloud tenancy — no prompts or documents leave your network boundary.
Fits: most privacy-sensitive mid-market companies. Cloud elasticity without third-party data exposure.
Hybrid, routed by sensitivity
A routing layer sends sensitive workloads to the private model and general work to a commercial API — one interface for users.
Fits: companies that want frontier quality where allowed and containment where required. Most common end state.
The decision rule. Classify your data first, then pick the posture — not the reverse. A one-page data classification (public / internal / confidential / restricted) written in an afternoon prevents both over-spending on air-gapped hardware for marketing copy and under-protecting payroll data on a public API.
Four layers, and less hardware than you think.
A private AI system is four layers: the model (open weights you download and own), the inference server that runs it (Ollama is the fastest on-ramp for a pilot; vLLM-class servers for production concurrency), the retrieval layer that lets it answer from your documents — policies, contracts, tickets — without retraining, and the access layer: single sign-on, per-role permissions, and audit logs of every prompt and source. Retrieval is where most of the business value lives; it is also where most shortcuts get taken.
WHAT RUNS ON WHAT
Small (7–14B params)
One workstation GPU (24 GB) — roughly a high-end desktop
Summaries, drafting, classification, extraction. Ideal pilot class; runs even on a strong laptop.
Mid (30–70B params)
One server with 2–4 datacenter GPUs, or equivalent VPC instance
Strong reasoning over retrieved documents, multi-step analysis, dependable internal Q&A. The production sweet spot.
Large (100B+ / MoE)
Multi-GPU node(s); meaningful capex or reserved cloud
Approaches commercial-API quality. Justified only after mid-class models prove volume and value.
Cost intuition. A credible offline pilot — one workstation, a small model, retrieval over a few thousand documents — is a five-figure project, not seven. Production for a few hundred users typically lands in the cost range of one mid-level hire. The expensive mistake is buying large-model hardware before a small model has proven the workflow.
From zero to a working private assistant.
Classify and choose. Write the one-page data classification, pick the posture (offline, VPC, or hybrid), and select one workflow where privacy currently blocks AI use — contract review, client-file Q&A, HR policy answers.
Stand up the pilot. A small open-weight model on one machine, retrieval over a curated document set (start with hundreds, not everything), and access for five named users. No integrations yet — prove answer quality first.
Evaluate like an operator. Fifty real questions from the workflow, graded by the people who own it: correct, sourced, useless, or wrong. Tune retrieval before considering a bigger model — most quality gaps are retrieval gaps.
Decide production. If graded accuracy clears your bar, size the production tier (SSO, audit logging, backup, a mid-class model if needed) with real usage data — not vendor benchmarks — and set the expansion list.
THE THREE PITFALLS THAT SINK PRIVATE AI
Boiling the document ocean
Indexing every share drive on day one buries good answers in stale duplicates. Curate a small, current corpus; expand deliberately.
Privacy theater
A private model with no access controls or logging just moves the risk inside. The access layer is part of the minimum build, not a later phase.
Nobody owns the box
Models update, indexes drift, GPUs fail. Name an owner and a maintenance cadence at go-live or the system quietly degrades into distrust.
Want a private-AI plan sized to your data?
A 30-minute diagnostic consultation — your data classes, the right posture, and a costed pilot plan within 48 hours.