AI Should Wait for the User, Not the Other Way Around

The Back Button That Waited for a Language Model

Here is a sequence of events that, on paper, contains nothing unusual.

A user opens FarmakoMed and taps into Health Vitals to glance at a blood-pressure trend. They read it, get distracted, and set the phone down. The screen turns off. Two minutes later they pick the phone back up, unlock it, and the app resumes exactly where they left it. So far, so ordinary. This is what every mobile app does dozens of times a day.

Then they press Back. They just want to return to the home dashboard. Nothing more.

And for a moment, not a long moment, but a noticeable one, the app doesn't go home. It hesitates. Somewhere beneath the surface, a local language model has begun warming up again after the resume, and an earlier version of our architecture had quietly decided that the whole application should hold still until that warmup finished. The user, who was not thinking about language models at all, experiences this as the Back button not working.

Technically, the app was doing precisely what it had been told to do. From the user's point of view, it was broken.

We didn't answer that question right away. We sat with it, because the honest answer turned out to be uncomfortable: navigation was waiting on the model because, somewhere early in the design, we had wired the two together without ever deciding to. This is the story of untangling them, and of realising that the fix wasn't a faster model. It was a different architecture.

The Assumption We Didn't Question

On-device AI has one property that cloud AI mostly hides from you: it has to start. A hosted model behind an API is, from your application's perspective, always on. You send a request, you get a response, and the machinery that keeps a model resident in memory somewhere in a data centre is somebody else's problem. There is no cold start you have to design around, because the vendor has amortised it across millions of requests.

A model running on the user's phone has no such luxury. The runtime has to be initialised. Weights have to be mapped into memory. Hardware acceleration, GPU or NPU, has to be set up. We call this step warmup, and on a phone it takes a real, perceptible amount of time. Not minutes, but not zero either. A few seconds, sometimes, depending on the device and what else it is doing.

When you are building an app whose signature features lean on a local model, recognising a medication from a photo, pulling an appointment out of a scanned letter, extracting an amount from a receipt, it feels natural, almost responsible, to make sure that model is ready before you let the user loose. So the startup sequence writes itself:

Application resumes, warm the AI, enable the application. Three boxes, one arrow between each. It is clean. It is easy to reason about. It has exactly one code path, and that code path guarantees that by the time a user can touch anything, every feature is fully armed. If you sketched this on a whiteboard in a design review, nobody would object. We didn't.

The flaw is not in any single box. It is in the arrows. This diagram silently asserts that the application is downstream of the AI runtime, that the app is not ready until the model is ready. And that assertion couples two systems that have no natural reason to be coupled. Reading a blood-pressure chart, scrolling a list of medications, navigating back to a dashboard: none of these need a language model. But the architecture had made all of them wait for one anyway.

Users Don't Think About AI

The most useful thing we did during this whole episode was not writing code. It was watching someone use the app and paying attention to the words in their head.

A user coming back to a health app after their screen has been off is thinking one of a small handful of thoughts. I want to go back. I want to see my medications. What time was that appointment? Let me just check the dose. Every one of these is a concrete, immediate intention about their own data. Not one of them is I hope the local language model has finished initialising its inference session. That sentence never occurs to anyone, because it belongs to our world, not theirs.

This is the gap that quietly produces bad experiences. Engineers carry a mental model of the system that is accurate, detailed, and completely invisible to the person holding the phone. We know there is a runtime. We know it warms up. We know warmup can fail. The user knows none of that and should never have to. When our internal model leaks into their experience, when the seams of our lifecycle become their waiting, we have handed them a problem that was only ever ours to solve.

Once we framed it this way, the Back-button problem stopped looking like a bug and started looking like a category error. We had let an implementation detail of how our AI happens to start become a fact about how our application behaves. The user's mental model was right and ours was wrong, not technically wrong, but wrong about what mattered.

We Were Optimising the Wrong Thing

The reflex, when a local model makes an app feel slow, is to make the model faster. It is a good reflex. There is an entire, satisfying discipline waiting for you the moment you decide the problem is performance: you measure warmup duration, you profile inference throughput, you watch memory high-water marks, you shave initialisation latency. All of it is real work, all of it is measurable, and all of it produces graphs that go down and to the right.

We did a fair amount of it. And it helped, a warmup that takes two seconds is genuinely better than one that takes five. But at some point we noticed that we were answering a question nobody had asked. The user pressing Back was not upset that warmup took five seconds instead of three. They were upset that warmup was happening in their way at all. Shaving two seconds off a wait the user should never have been placed in front of is optimisation aimed in the wrong direction.

That sentence marks the exact point where the project changed character. We had been treating this as a performance-engineering problem, make the slow thing fast. It was actually an experience-engineering problem, make sure the unavoidable cost lands where nobody is standing. And the moment you accept that framing, faster inference stops being the goal. The goal becomes controlling when the cost is paid, which is not a question about the model at all. It is a question about architecture.

Two Independent Lifecycles

The redesign came down to a single idea, and like most useful architectural ideas it sounds obvious once said out loud: the application and the AI runtime are two different systems, with two different lifecycles, and neither one owns the other.

The application has a lifecycle it has always had. It comes to the foreground and goes to the background. It navigates between screens. It renders. It reads and writes through its repositories. This lifecycle is driven entirely by the user and the operating system, and it has existed, complete and correct, since long before we added a single model to the app. It does not need AI to be part of it. It never did.

The AI runtime has its own, quite separate lifecycle. It is cold, not yet loaded. It becomes preparing when warmup begins. It reaches ready when the runtime is initialised and can serve a request. And it can land in failed when something goes wrong, a device that can't support it, memory pressure, a runtime error. This lifecycle is driven by resource availability and by explicit requests for AI capability. It has nothing to say about whether the user can press Back.

The two systems still talk to each other, of course they do. The application can ask the runtime to begin preparing. The runtime can tell the application when it has become ready, or that it has failed. But this is communication, not ownership. A message passes between two peers. Neither one blocks on the other's internal state as a precondition for doing its own job. The application renders and navigates whether the runtime is cold, preparing, ready, or failed. The runtime warms up whether the user is on the home screen or deep in a settings menu. Their lifecycles overlap in time without being fused in code.

This is the heart of the article, and it is worth stating as plainly as possible: the application should be considered ready the moment the user can interact with it, not the moment every subsystem it might eventually call has finished initialising. Readiness is about the user's ability to act, not about the completeness of the machine behind the curtain.

AI Becomes a Service, Not a Prerequisite

Separating the lifecycles forced a shift in how we thought about what AI even is, architecturally, inside the app. In the old model, AI was foundational, a layer the rest of the application was built on top of, such that if the foundation wasn't poured, nothing above it could stand. In the new model, AI is a service the application can call when it needs one. That is a demotion, and it is the correct demotion.

The useful comparison is to the other capabilities a phone app already treats this way. Think about the camera. The GPS. Biometric authentication. No competently built app blocks its entire interface at launch waiting for the camera to be ready, or refuses to render until it has a GPS fix, or freezes until the fingerprint sensor has warmed up. These capabilities are requested when a feature needs them, and only then does the user encounter whatever small readiness cost they carry, the camera taking a beat to open, location taking a moment to resolve. The rest of the time they sit dormant and unmissed.

Reframed this way, the old philosophy, if AI isn't ready, the application isn't ready, reads almost absurdly. We would never say if the camera isn't ready, the application isn't ready. We would never gate the home screen on a GPS lock. The only reason we had done the equivalent for AI is that AI arrived feeling special, feeling central, feeling like the point of the product rather than one capability among several. It is a capable and important part of FarmakoMed. It is still just a capability, and capabilities are requested.

Invisible AI

What does this feel like, from the outside, once it's built? Ideally, like nothing at all.

The blocking overlay is gone. There is no frozen screen while the runtime spins up, no full-page loader standing between the user and their own data, no modal explaining that AI is preparing and would they please wait. In its place there is, at most, a small, quiet status indicator, a slim banner that says something like Preparing AI… and otherwise stays out of the way. The user keeps navigating straight through it. Home opens. Vitals open. Settings open. A couple of seconds later the banner changes to AI ready, or simply disappears.

Most people never consciously register any of it. And that, the not-registering, is the entire point, not a disappointing side effect of it. We are so conditioned to want our work to be seen that it takes a deliberate act of engineering maturity to build something whose success is measured by how completely it goes unnoticed.

There is a genuine design tension hiding in that little banner, and it's worth being honest about. Show too much, and you've reintroduced the very anxiety you were trying to remove, now the user is watching a progress indicator for a thing they didn't ask for, which is arguably worse than a clean wait. Show nothing, and on the rare occasion the user does immediately reach for an AI feature, they're left wondering why it isn't responding. The banner is a deliberately minimal answer: present enough to explain a momentary delay if the user happens to run straight into one, quiet enough to be ignored completely by everyone who doesn't. Invisible does not mean absent. It means proportionate.

The Work Hiding Behind a Banner

It would be dishonest to describe this as a small change. From the user's side it is a banner replacing a blocker, trivial, barely a feature. From the engineering side it was a significant amount of architecture, and it is worth being specific about what "just decouple them" actually costs, because the gap between how small the change looks and how much work it is turns out to be one of the real lessons here.

Splitting one lifecycle into two means the AI runtime now needs a lifecycle of its own that is explicitly modelled rather than implied, a small state machine with real, named states and legal transitions between them.

Each transition in that diagram is a design decision that the old serial chain let us skip. Who triggers preparing, the app on resume, a preload heuristic, or the first real request? What happens to in-flight requests when ready falls back to cold because the operating system reclaimed the memory? How many times, and how fast, do we retry from failed before we stop and simply tell the affected feature that AI is unavailable right now? None of these questions exist when there is one blocking path. All of them exist the moment the runtime has a life of its own, and answering them well is the actual work.

And all of it has to hold across two platforms whose lifecycle and memory-management models are not the same. Android and iOS make different promises about when a backgrounded process keeps its memory, when it is frozen, and when it is killed outright. A runtime lifecycle that is genuinely independent of the application lifecycle has to survive all of those platform behaviours without leaking a single one of them back to the user as a stalled screen. This is precisely the sort of concern that a cloud architecture never has to think about, when the model lives in a data centre, the phone going to sleep is not the model's problem. On-device, it is very much the model's problem, and therefore ours.

None of this is exotic. There is no novel algorithm here, no research result, nothing we would claim as a breakthrough. It is ordinary, careful systems engineering in service of a wait that the user will, if we did it well, never even know we removed. That is roughly the least glamorous description of high-value engineering we can think of, and we mean it as a compliment to the category.

A Lesson Beyond AI

Strip the specifics away and this stops being a story about local language models at all. It is a story about ownership, and about the way early architectural assumptions harden into invisible law.

When we started, coupling the application's readiness to the AI's readiness felt not just reasonable but responsible. Of course the app should wait for its core capability. The reasoning was sound given the frame we were in. The problem was never the reasoning. It was that the frame itself had gone unexamined, and stayed unexamined precisely because it worked well enough on the paths we happened to test. The most dangerous architectural decisions are rarely the ones that are obviously wrong. They are the ones that are quietly, defensibly right on the happy path, and that nobody ever revisits because there was never an error message forcing them to.

So the transferable questions are not about AI. They are these. What responsibilities have you assigned to a subsystem that actually belong to the application, or the other way around? Which of your components block on each other not because the domain requires it but because that is how the code grew? What in your architecture is the way it is simply because "that's how we started," and has never once been asked to justify itself since?

We suspect most non-trivial systems are carrying at least one dependency like our old one: technically defensible, quietly costing something, and load-bearing only because no one has questioned it. Ours happened to surface as a Back button that hesitated. Yours will surface as something else. The pattern, a subsystem's implementation detail leaking out as a fact about the whole system's behaviour, is what to watch for.

Conclusion

Performance matters. Memory usage matters. Inference speed matters, and we will keep working on all three, because on a phone every one of them is a real constraint with real consequences. Nothing here is an argument against optimisation.

But this whole episode was a reminder that architecture is what determines whether the user ever encounters those constraints in the first place. Users do not benchmark inference. They do not know or care whether warmup takes two seconds or five. What they remember is the interruption, the moment the phone didn't do the simple thing they asked, because something they never requested wasn't finished yet. You can win every performance graph and still lose that moment, and the user will only remember the moment.

The fix, in the end, was not a faster model. It was refusing to let the model's schedule become the user's schedule. Two lifecycles instead of one. A capability instead of a prerequisite. A quiet banner instead of a locked screen. And underneath the banner, more engineering than the banner will ever admit to.

If there is one idea worth taking from this, it is that last one, not because it is clever, but because the inverse is so easy to build by accident. Sometimes the most important thing engineering can do is not make the software faster. It is to make the waiting disappear from where the user is standing, and quietly move it somewhere they will never look.

One Last Thought

Great infrastructure disappears. Great user experience disappears. When both are working, the person holding the phone presses Back, goes home, and thinks about nothing at all, which is exactly as much as they should ever have to think about the language model that was, a second earlier, still waking up. That silence is the deliverable. Everything else is how you earn it.