Battery-Aware AI Inference on Mobile

The Metric Nobody Benchmarks

When engineers compare on-device language models, they compare the things that show up in a benchmark table: accuracy, latency, model size, RAM footprint, tokens per second. These numbers are easy to measure, easy to publish, and easy to argue about on a leaderboard.

Users compare something else. Battery.

Nobody who uses a medication app every day thinks "inference took 800 milliseconds instead of 600." They think "my phone is warm and at 40% by three in the afternoon, and I'm not sure why." If the answer turns out to be an app they open for thirty seconds a day, the relationship with that app changes, quietly, and usually permanently, because a battery complaint rarely comes with a stack trace. It just comes as an uninstall.

This is the argument of this article: for local AI on a phone, battery is not a metric you optimise after the feature ships. It is an architectural constraint that shapes the feature from the first design decision, where the model lives, when it loads, when it releases, and what it's allowed to do without being asked. Everything below is grounded in the actual on-device AI architecture running in FarmakoMed today, not in a hypothetical one.

Every Token Has an Energy Cost

Start from the physics, briefly, because it explains why this problem exists at all. Inference is computation, matrix multiplications, memory reads, weight lookups, repeated for every token a model produces. Computation consumes energy. That has always been true of AI; what's changed is where the energy is spent.

Cloud AI hides that cost inside a data centre. When a cloud model answers a prompt, the electricity is drawn from a rack thousands of kilometres away, on a grid connection sized for exactly this kind of load, cooled by infrastructure built for exactly this kind of heat. The user's device does almost nothing: it sends a short request and renders a response. The energy story is somebody else's problem, by design.

Local AI removes that distance. The computation moves from the data centre into the user's pocket, and so does the energy bill. A phone's CPU, GPU, or NPU does the same class of work a cloud server would have done, but on a battery designed to last a day of ordinary use, with no active cooling beyond a thin aluminium chassis. There is no separating "the AI feature" from "the power budget" anymore, because they now physically share the same tiny, thermally-constrained, battery-limited machine as every other app the user has open.

This doesn't mean the answer is "make the model as small as possible and stop there." Model size, quantisation, and hardware acceleration (CPU vs. GPU vs. NPU) all matter, and they're the part of this story that gets the most attention because they're the easiest to benchmark. But they're not the whole story, and for a feature people open for seconds at a time, they're arguably not even the largest part of it. A model can be extremely efficient per token and still drain a battery all day, if it's never allowed to stop running.

Why Healthcare Applications Are Different

Battery tolerance isn't uniform across app categories, and it's worth being explicit about why a medication app sits at the strict end of that range.

A game is opened for an engaged session and closed. A photo filter runs for a few seconds and produces a result the user was actively watching for. A health app is different in a way that changes the engineering brief: it's installed once and expected to stay installed for years, it's consulted in short, unpredictable bursts throughout the day rather than in long sessions, and, critically, it's expected to be quietly, reliably there the one time it matters, whether that's reading a prescription label at a pharmacy counter or pulling up emergency information for someone else. None of that tolerates a reputation for draining the battery.

That combination, long install lifetime, low daily engagement time, high reliability expectation at the moment of use, means a health app has the least room of any category to treat background resource usage as someone else's problem. Users don't uninstall a game over battery; they might just play it less. They uninstall a health app over battery, because the app was supposed to be invisible infrastructure, and invisible infrastructure that visibly drains your phone has broken its half of the bargain.

Battery Is a UX Feature

Here is the idea this whole article is really about, and it's worth stating plainly rather than building up to it.

Users never file a bug that says "the inference engine consumed too much power." They don't have that vocabulary, and they shouldn't need it. What they experience, and what they act on, is much simpler: this app drains my battery. It's a UX judgement, not a systems judgement, and it gets made regardless of whether the user could ever explain why it's true.

That framing has a direct engineering consequence: good energy engineering is measured by its absence from the user's attention, not its presence in a settings screen. A battery graph, a "low power mode for AI" toggle, a warning dialog about model warmup. These are all admissions that the engineering didn't finish. The goal isn't to explain the energy cost to the user well. It's to make the explanation unnecessary, by keeping the cost small enough and rare enough that it never becomes something the user has to reason about at all.

This is the central claim worth carrying into every other section below: battery isn't a performance metric alongside latency and accuracy. It's a product experience, sitting on the same list as "does the camera focus quickly" and "does the app crash." Users don't grade it separately. They fold it into whether they trust the app at all.

The Engineering Trade-Offs

Once battery is treated as a real constraint, a familiar pattern shows up: almost every lever that improves one AI metric quietly worsens another, and battery is very often the one that absorbs the difference.

• Lower latency is usually bought by keeping the model resident in memory and the runtime "hot," ready to answer instantly, which means higher standby consumption, because something is being kept alive that the user isn't using.
• Faster startup is usually bought by preloading or caching more state ahead of time, which means more memory retained, competing with everything else the OS wants that RAM for.
• Continuous readiness, a model that's always warm, "just in case", trades directly for higher battery usage, whether or not the feature is ever used that day.

None of these trades is a bug. Each one is a legitimate choice for some product. A voice assistant that's expected to respond the instant you speak has good reasons to stay warm. The point isn't that keeping a model resident is wrong. It's that it's a decision with a battery price tag attached, and that price tag needs to be paid deliberately, for a feature that's used often enough to justify it, rather than by default because nobody modelled the cost.

Lessons from FarmakoMed

The clearest way to make this concrete is to describe where FarmakoMed's own on-device AI architecture actually landed, and why. FarmakoMed runs a local vision-language model (Gemma, served through Google's LiteRT-LM runtime) for tasks like reading a photographed medication package. That model is a multi-gigabyte artifact sitting in a phone's RAM the moment it's loaded, which is precisely the profile that makes "when is it loaded" the single most consequential battery decision in the whole feature.

The runtime doesn't stay warm in the background, on purpose. The moment the app leaves the foreground, the AI runtime is released: the model is unloaded and its memory freed, rather than kept resident "just in case" the user comes back needing it instantly. That single rule removes the largest and least justifiable source of drain outright, an idle, backgrounded app has no legitimate reason to keep a multi-gigabyte model warm, because it isn't answering anything for anyone while it's backgrounded.

Coming back doesn't mean an instant reload, either. On resume, the app deliberately waits briefly, on the order of a second and a half, before re-warming the model, rather than racing to reload the instant the app becomes active. That pause exists so the OS has time to finish reclaiming the memory that was just freed, instead of the app immediately fighting the system for the same RAM it just gave back. It's a small, deliberate delay that trades a sliver of perceived snappiness for a cleaner, more reliable reload, and, not incidentally, avoids the wasted energy of a reload that has to be retried because it started too early.

The model also steps aside for other power-hungry hardware. Opening the camera to photograph a medication box is itself a real power draw, the sensor, the image signal processor, the preview pipeline. Rather than let a resident multi-gigabyte model sit in memory at the same moment the camera stack spins up, the app proactively releases the AI runtime immediately before the camera opens, and reloads it afterward. Two expensive subsystems simply aren't allowed to be resident at once if one of them can wait its turn.

Readiness is requested, not assumed. The AI runtime doesn't warm up the instant the app launches, on the assumption someone might use it. It warms up when a feature that actually needs it is invoked, or after the app is confirmed active and stable, and every request for warmth is deduplicated, so an eager user tapping the scan button repeatedly doesn't trigger repeated, wasted warmup work underneath a single visible spinner.

The two platforms even spend their warmup energy differently, on purpose. On Android, "warming up" loads the runtime and engine without running a real inference through it, a deliberately cheap wake-up. On iOS, warmup runs a small real inference (a short, throwaway prompt) rather than just loading the engine, specifically so a broken model or tokenizer is caught immediately rather than surfacing on the user's first real request. That's a genuine, platform-specific trade: iOS spends a little more energy up front in exchange for catching failures before the user ever sees one; Android optimises the same moment for the cheapest possible wake-up. Neither is "wrong". They're different answers to the same question, chosen for different runtime maturity on each platform.

Nothing about this touches the operating system's own background execution. The only background job the app schedules is a resumable model file download, the kind of long-running transfer that legitimately benefits from surviving app-process death. Actual inference, OCR, and document analysis only ever run while the app is in the foreground, in direct response to something the user asked for. There is no scheduled job anywhere that wakes the AI engine on a timer to do work nobody requested, which is, in practice, the single biggest lever available: the cheapest inference is the one that never runs unobserved.

Individually, none of these are exotic techniques. Release what you're not using. Don't race the OS for memory. Don't let two expensive things run at once. Ask before you assume readiness is needed. Together, they describe an architecture where the AI model's default state is off, and every second it spends resident in memory has to be justified by an actual, present user action, which is exactly the posture a feature used for seconds a day, for years, needs to earn any trust at all.

Modern Smartphones Are Changing

It's worth being honest that some of today's compromises are temporary, because the hardware underneath them is moving quickly. Dedicated NPUs are becoming standard rather than exceptional. Mobile GPUs keep closing the gap with what used to require a discrete accelerator. Runtimes like LiteRT are under active development specifically to squeeze more useful work out of the same silicon and the same battery. Quantization techniques keep improving, letting larger, more capable models fit in the memory and power envelope that used to only accommodate much smaller ones.

None of that is a reason to defer the architectural discipline described above until "the hardware gets good enough." Better hardware raises the ceiling on what's affordable to run continuously; it doesn't remove the cost of running things continuously that don't need to be. A model that's twice as energy-efficient per token, kept resident and warm all day out of habit, can still lose to a less efficient model that's simply asleep most of the time. Today's compromises about model size and warmup strategy may well disappear in a generation or two of hardware. The discipline of asking "does this need to be running right now" won't. It just gets to be applied to bigger, more capable models.

Invisible Engineering

There's a pattern worth naming here, because it shows up well beyond battery, and it's the closest thing this article has to a single takeaway.

Users almost never notice excellent resource engineering. They only notice its absence. Nobody praises an app for a battery they didn't have to think about, a day that simply passed, phone in pocket, no anxiety about the percentage in the corner. The engineering that produced that non-event is invisible by definition: if it worked, there is nothing to see, nothing to report, nothing to review favourably. The only trace it leaves is the trace it prevented.

That's an uncomfortable property for engineering work to have, because it means the discipline described in this article will rarely be celebrated on its own terms. Nobody writes an app-store review that says "the AI runtime lifecycle correctly released memory on backgrounding." What they write, if the engineering fails, is "this app kills my battery." The absence of that sentence is the actual deliverable, and it's worth treating it as one, with the same intent and review rigour as any feature that does show up on a screen.

Conclusion

Local AI is a genuine architectural shift, and not only for privacy reasons. It relocates the energy cost of intelligence from a data centre built for that exact purpose to a battery in someone's pocket, built for an ordinary day of ordinary use. That relocation means battery can no longer be treated as an optimisation pass that happens after a feature ships. It has to be a constraint the architecture is built around from the start: when the model loads, when it's allowed to stay resident, what has to ask permission before it runs, and what's never allowed to run unobserved.

For a healthcare application in particular, installed for years, opened for seconds, trusted to be ready at moments that matter, that discipline isn't optional polish. It's part of the basic contract of being trustworthy software that stays on someone's phone.

One Last Thought

The best AI running on a phone isn't the one that generates the most tokens. It's the one users forget is even there, because it only ever spent energy on something they actually asked for, and gave every other second of the day back to the battery.