Shrinking the model to fit a budget phone

A speech model that’s good enough to be worth shipping is, by default, big. A phone that costs a few thousand rupees and has only a sliver of RAM to spare is not built to carry that kind of weight — the operating system, betterflo itself, and whatever else is open all need a share of the same small pool of memory. If the model doesn’t fit, none of the accuracy work from getting it good matters, because it never loads in the first place.

This is the constraint we designed for from day one, not a workaround we bolted on after the fact. Betterflo tiers what it runs by what the device can actually do — RAM and CPU headroom — rather than assuming everyone is carrying this year’s flagship. On the higher tiers, a phone with real memory to spare can run a larger, more accurate model. On the entry tier — the one with the least RAM to spare, see /devices for the exact range — the model has to be small enough to load, stay loaded, and still run fast enough that dictation feels instant rather than laggy.

The main lever for getting there is quantization: storing and computing with the model’s weights at lower numeric precision instead of full precision. Done carelessly, that’s a shortcut that costs accuracy. Done deliberately — evaluated against the full-precision version rather than assumed to be “close enough” — it’s a real trade a team can defend: a meaningfully smaller memory footprint and faster load, for an accuracy cost we check rather than guess at. The gate is simple: if quantizing the model measurably breaks it, it doesn’t ship that way, no matter how much RAM it saves.

There’s a second reason this matters beyond raw model size: voice dictation is a service that has to be ready the instant you tap the mic — whether that’s the floating mic over another app or the keyboard. A model that’s too large ends up swapped out of memory the moment you switch apps, which means every dictation starts with a reload — the opposite of what dictation is for. So fitting the entry tier isn’t just about clearing an install-time check; it’s about staying resident and responsive through ordinary, distracted phone use, in a browser tab, a chat app, a dozen other things half-open at once.

None of this is meant to be a permanent ceiling. As device tiers shift and more people carry more capable phones, the floor moves up too. But the discipline stays the same either way: we don’t get to claim a model “runs on-device” using a device most of our actual users don’t own. If it only runs on the phone we wish everyone had, it doesn’t run for India — and shrinking the model to fit the phone that’s actually in someone’s hand is the unglamorous engineering work behind a promise we’re willing to put on the pricing page.