Language models running locally, with no data leaving the network

Context

Companies that want to use AI on internal documents — contracts, technical sheets, system logs — almost always get stuck in the same place: the data is not allowed to leave for an external provider.

Problem

The default assumption is that you need data centre hardware. You don’t, but the out-of-the-box configuration of self-hosting tools produces results so poor that people give up and conclude local models don’t work.

What I did

1. 01A full stack on a 12 GB consumer GPU: twenty-billion-parameter models at roughly 30 tokens per second
2. 02Tuned context and cache — the default setting truncates the instructions and makes the model look incompetent
3. 03Service exposed on the local network only, with no public access
4. 04Wired into an agent running on cron: health checks, nightly backup with retention, internet link monitoring, video event analysis
5. 05Reports pushed automatically to Discord, with no human in the loop