Local by default, and what that actually costs

Two of the four things I've shipped run a neural model on the user's own hardware. iRead does text-to-speech entirely on the phone. Anelo runs whisper for speech recognition and qwen2.5 for cleanup, both on your laptop.

Neither of those is the easy choice. Calling an API would have been faster to build, smaller to ship, and better on old hardware. Here's the actual trade, in both directions.

What it costs

The app gets bigger, or the first run gets slower. A neural voice is not a small file. You either ship it inside the app and watch the download size climb, or you fetch it on first launch and explain to the user why the thing they just installed isn't ready yet. Both options are worse than a 40KB API client.

Old hardware suffers. A server-side model performs the same for everyone. An on-device model performs like the device it's on. The gap between a current phone and a five-year-old one is the kind of thing you can't fix in software, and it shows up as a support email that begins "is it supposed to be this slow".

You can't improve it behind their back. Swap the model on a server and every user gets the better version that afternoon. On-device, improvements ship at the speed of app updates, which is to say at the speed of app review plus however long people take to update.

Debugging is harder. When transcription goes wrong on someone's machine, I can't look at the input. I don't have it. That's the entire point, and it is also genuinely inconvenient.

What it buys

It works with no network. iRead reads aloud on a plane. Anelo transcribes in a basement. This isn't a marketing bullet — it's the difference between a tool you can rely on and one that has moods.

Inference costs nothing per user. This is what makes iRead a one-time purchase instead of a subscription, and Anelo free instead of freemium. There is no per-request cost quietly accumulating in the background that has to be recovered from somebody. The pricing follows from the architecture.

Privacy stops being a promise. This is the part that actually matters. A privacy policy saying "we don't store your audio" is a statement about intent, retention schedules and access controls — all things you're asked to take on faith. "The audio never left the machine" is a statement about network traffic, and anyone sufficiently annoyed can verify it with a packet capture.

Those are different kinds of claim. One depends on my continued good behaviour, my hosting provider's, and that of whoever might acquire either of us. The other doesn't depend on me at all.

The rule that makes it real

None of this means anything if the software quietly falls back to a server when the local path is inconvenient. So Anelo has one rule:

If the local models aren't ready, it fails. It does not silently go to the cloud.

Cloud transcription is available — Groq, OpenAI, Gemini, with your own API key — and for some people it's the better trade. But it is off unless you turn it on, and turning it on is an explicit choice made in settings, not a fallback that happens on your behalf while you're not looking.

A default that quietly stops applying under load isn't a default. It's a marketing claim with an escape hatch.

Where the line sits

I'm not dogmatic about this. MindBloom talks to a server, because a journal companion that carries months of context genuinely needs a model larger than anything that runs comfortably on a phone. WearWhat generates images, which is not happening locally in 2026 at any quality worth shipping.

The rule isn't "everything on-device". It's: run it on the device when the quality is good enough there, and say plainly where it runs when it isn't.