Private AI Appliance
On-Site AI. Local by Default.
Hardware in your building, running local inference — with a routing policy that only reaches out to the cloud when a task genuinely needs it, visibly rather than silently.
Why On-Prem
For Data That Shouldn't Leave the Building
Some businesses cannot or will not send certain information to a third-party model — customer records, financials, anything under a contractual or regulatory constraint. This offering exists for exactly that case, without giving up AI assistance entirely.
Local by Default
No data leaves the building unless the routing policy says so.
Hybrid Routing
Cloud models handle only the tasks that genuinely need them.
Optional Voice
A voice interface for structured, everyday requests.
Visible, Not Silent
How Hybrid Routing Actually Decides
The same routing discipline we sell is the one we run. Every request is handled by the simplest capable model — and which one handled it is never a mystery.
| Example Request | Handled By |
|---|---|
| Looking up a customer record or internal document | Local |
| Drafting a routine internal email or summary | Local |
| Complex reasoning over a long, unfamiliar document | Cloud (visible) |
| Voice command to check status or trigger a known workflow | Local |
The exact policy is configured per install — what counts as sensitive, which tasks are allowed to escalate, and what happens when the local model is uncertain are all decisions we make with you, not for you.
Reference Architecture
Hardware Specification
The install we productized: Mac Mini–class host with local models, integrated backup, and optional voice edge. This is the documented reference, not a locked SKU — we configure around your existing infrastructure when it makes sense.
Primary Host (Reference)
- Platform
- Mac Mini M4 Pro
- Memory
- 24GB RAM
- Internal storage
- 512GB SSD
- External backup
- 2TB SSD (dock)
Always-on headless server. SSH and LAN-bound inference. The 512GB internal houses runtime; the 2TB external handles backups, exports, and archives.
Internal Space Budget
- macOS + apps
- ~80–120GB
- Local models (1–2)
- ~15–40GB
- Runtime memory
- Grows over time
- Working headroom
- ≥100GB free
Heavy archives and model archives live on the external drive when space tightens. Runtime stays on internal for speed.
Optional Voice Edge
For voice-controlled workflows, a Raspberry Pi 4 running Home Assistant OS with Wyoming add-ons handles speech-to-phrase and text-to-speech locally, forwarding conversation to the Mini's LAN-bound model. Not required for the core install; add only if you need hands-free, voice-first interaction.
Implementation Path
Install Milestones
The documented runbook, organized into customer-facing phases. Each step builds on the last; the whole install is reproducible and version-controlled as config-as-code.
Hardware Prep
macOS config for always-on headless operation, SSH access, Homebrew, external backup volume mounted and formatted.
Local Inference + Agent
LM Studio with a 24GB-appropriate model (65k context), Hermes Agent installed with SOUL/USER config, first conversation verified.
Hybrid Routing Config
Cloud model keys wired (Nous Portal + direct providers), routing policy documented and live, usage tracking enabled.
Messaging Interface
Slack or Telegram gateway set up, launchd service for auto-restart on crash/reboot, first mobile message round-trip confirmed.
First MCP Integrations
Filesystem, git, and at least one business tool (e.g. GitHub) wired and verified — the foundation for tool orchestration.
Backups & Always-On
Nightly rsync to the external drive via launchd, rotating retention, manual restore tested. The hub survives a full internal wipe.
Optional Voice
Pi4 with Home Assistant OS, Wyoming speech add-ons, LM Studio bound to the LAN, first voice-to-action round-trip working.
Tool Orchestration
Integration Catalog
MCP (Model Context Protocol) unifies tool access across business systems, smart building, and memory — one routing layer instead of one-off integrations per tool.
Core
- Filesystem (local workspace, config repos)
- Git (status, diff, log, push/pull automation)
Business
- GitHub (issues, PRs, repo file access)
- Slack (send, schedule, search channels)
- Google Workspace (Gmail, Calendar, Drive, Tasks)
- QuickBooks (invoices, customers, read-only by default)
- Vercel (deployments, logs, analytics)
Smart Building (Advisory-Only)
- Home Assistant (read device state, suggest actions — never writes to devices unattended)
Device control stays strictly advisory. The agent can flag and suggest, never execute a state change without explicit approval.
Memory & Continuity
- Memory server (cross-session knowledge graph, persisted locally)
- ME profile (personal facts, preferences, durable identity)
Models are interchangeable workers; the brain lives in MCP memory, not any one model's context. Switch models without losing continuity.
Routing Discipline
Sensitive-Local-By-Default Policy
Which tasks stay on-device and which escalate to cloud models — the documented policy generalized from our own hub install for SMB use cases.
| Task Involves… | Route To |
|---|---|
| Client data, consulting records | Local |
| Financial data (invoices, QuickBooks) | Local |
| Personal PII | Local |
| Hard reasoning, large context, no sensitive data | Cloud (visible) |
| Uncertain / mixed | Ask first |
The exact definitions (what counts as "sensitive," which tasks may escalate, whether the local model should ask before routing uncertain cases) are decisions we document with you during install, not defaults we impose.
Proven on Our Own Infrastructure First
This is not a vendor pitch for hardware we have never run. The install runbook, routing policy, and tool-integration catalog behind this offering come directly from our own on-site AI hub — see the case study for how it actually works day to day.
Read the Hermes Hub case study