On-device AI on phones and wearables

By Marco Lopes · July 22, 2026 · Mobile & Flutter

On-device AI changes where we draw the boundary between an interface and its intelligence. Instead of sending every image, audio segment, or sensor reading to a server, we can process selected inputs directly on a phone or wearable. That can make interactions faster, keep features useful without connectivity, and reduce how much sensitive information leaves the device.

At Kordspace, we approach this as a product and systems-design decision, not a requirement to run the largest possible model locally. As a remote collaborative software, AI, Web3, hardware, and design studio, we work across the boundaries these products expose: mobile interfaces, native runtimes, physical sensors, and cloud services.

The useful question is not whether a device can run AI. It is whether we can ship a reliable feature that earns its memory, battery, and maintenance costs.

Start with a task, not a model

We first define the smallest decision or transformation that would improve the experience. Recognizing a gesture, identifying a document boundary, detecting a spoken command, and summarizing a long conversation are very different workloads. Treating them all as a chatbot problem usually adds unnecessary complexity.

For a wearable companion app, we might need to distinguish a deliberate movement from ordinary walking. A compact classifier working on short sensor windows may be more appropriate than a generative model. For a phone-based field tool, local text recognition followed by deterministic validation may solve the problem without open-ended generation.

We write the acceptance criteria before choosing the runtime: acceptable mistakes, response time, offline requirements, supported devices, and behavior when the system cannot decide. We also build a non-AI baseline. If a threshold or straightforward rule performs adequately, we keep the simpler solution.

Decide what belongs on each device

“On-device” does not identify a single execution environment. A smartwatch, a low-cost Android phone, and a recent flagship have different compute, memory, and background-execution constraints.

We usually divide the system into three possible tiers:

• Wearable: capture signals, perform lightweight filtering, and detect simple events when hardware and platform permissions allow.
• Phone: run more demanding classification, recognition, or generation, and coordinate the interface.
• Cloud: handle optional tasks that exceed local capabilities, plus training, evaluation, and controlled distribution.

That split is not mandatory. An offline product may deliberately omit cloud inference. A safety-sensitive feature may need to remain functional without a paired phone. We document those boundaries rather than assuming that connectivity will rescue a weak local design.

For wearables, we also evaluate the energy cost of communication. Constantly streaming raw sensor data can undermine savings from lightweight local processing. Batching, event-triggered transfers, and compact feature representations deserve measurement alongside inference itself.

Match the runtime to the workload

For Flutter applications, we keep the interface portable while choosing an inference path that fits each target platform. Depending on the workload, that may mean Core ML on Apple devices, LiteRT on supported mobile targets, or another maintained native runtime with the operators and acceleration we need.

We do not assume that a model working on a desktop will convert cleanly or run efficiently on a phone. Unsupported operators, tensor shapes, preprocessing differences, and accelerator compatibility can change the result substantially.

Before committing, we build a thin native prototype and test a representative model on real hardware. We compare CPU execution with available accelerated paths because acceleration can introduce setup and data-transfer overhead. A GPU or neural accelerator is not automatically the fastest choice for a small, intermittent task.

We also review runtime licensing, model licensing, binary size, maintenance activity, and platform support. For operating-system-provided AI features, we verify current device, language, permission, and availability requirements instead of treating them as universal capabilities.

Keep Flutter responsive and the inference layer replaceable

We structure the Flutter app around a narrow inference interface. The UI supplies a defined input and receives a structured result, timing information, and an explicit outcome such as success, unavailable, or uncertain.

A practical flow looks like this:

capture → validate → preprocess → infer → postprocess → present

Each stage gets a clear owner. Image orientation, audio sample rate, normalization, and sensor units belong in a tested preprocessing contract. Without that contract, a technically correct model can produce incorrect results because the application feeds it different data from its training pipeline.

We use platform channels or FFI where appropriate to reach native implementations. We move substantial Dart-side processing off the main isolate, while remembering that an isolate does not automatically make native inference asynchronous. Threading, cancellation, and runtime thread safety still need explicit handling.

For camera or sensor streams, we apply backpressure. Processing the latest useful frame is often better than queuing every frame and returning increasingly stale answers. We bound buffers, release native resources predictably, and avoid unnecessary copies of large inputs across language boundaries.

Budget memory, battery, and heat together

A model that runs once is not necessarily ready for production. We measure cold initialization, warm inference, sustained operation, peak memory, and recovery after the app moves between foreground and background.

Model file size is only part of memory consumption. Working tensors, image buffers, runtime allocations, and generation caches can dominate the actual footprint. Loading multiple models simultaneously may also create pressure that a single-model benchmark misses.

We test extended sessions because thermal throttling can change both latency and user comfort. On wearables, a workload that looks modest in isolation may compete with sensing, display updates, connectivity, and other system functions.

Our first optimizations are usually architectural: infer less often, crop inputs, shorten windows, reuse initialized resources, and stop work when results are no longer useful. We then consider quantization, smaller architectures, or distillation. Every optimization goes back through quality evaluation; a smaller model is not an improvement if it fails the product’s most important cases.

Treat local processing as one privacy control

Keeping inference local can reduce data exposure, but it does not make an application private by default. Sensitive inputs can still leak through analytics events, crash reports, debug logs, temporary files, backups, or an unexpected fallback request.

We map the complete data lifecycle: what we capture, where we process it, how long we retain it, and what leaves the device. We prefer ephemeral processing when storage is unnecessary and use platform-appropriate protected storage when retention is justified.

Permissions should match visible features. We explain why microphone, camera, or motion access is needed, and we provide useful behavior when permission is denied. We also distinguish operating-system permission from consent to transmit data to a service.

If a cloud path exists, we make its boundary explicit. A claim such as “your recordings stay on your device” cannot coexist with silently uploading recordings when local inference fails. Our product copy must describe the system we actually ship, including optional diagnostics and support workflows.

Use the cloud deliberately, not automatically

Our preferred cloud foundation is Google Cloud, while AWS and Azure remain valid choices when requirements or existing infrastructure favor them. For an on-device product, cloud value often comes from the supporting lifecycle rather than from handling every prediction.

We can use managed training and evaluation workflows, object storage for versioned model artifacts, and authenticated services for configuration or optional inference. On Google Cloud, that may involve Vertex AI, Cloud Storage, and Cloud Run, with responsibilities separated rather than bundled into a single service.

Cloud fallback needs a policy. We decide which inputs may leave the device, whether users must opt in, how requests authenticate, and what happens during timeouts. We never embed privileged service credentials in a mobile application.

We also avoid silent semantic changes. If local and cloud models behave differently, the interface and downstream logic must tolerate that difference. An offline result should not become misleading simply because a more capable remote model exists.

Design for uncertainty and interrupted work

AI output should not force us into pretending that every answer is dependable. We design explicit states for unsupported hardware, missing model files, insufficient input, cancellation, and low-confidence results.

For classification, we validate confidence behavior against representative data rather than assuming that a score is a calibrated probability. For generative features, we do not display a made-up certainty percentage. Instead, we constrain the task, validate structured output where possible, and make important results reviewable.

The best fallback may be manual entry, a clearer capture prompt, or a conventional interface, not a second model. We preserve the user’s work when they switch apps or lose the phone connection, and we avoid promising indefinite background execution on platforms that restrict it.

For health-related wearable experiences, we draw a careful line between wellness feedback and medical claims. Higher-risk uses require domain validation and an appropriate regulatory assessment, not just a disclaimer beneath an attractive chart.

Ship models like versioned software

A model update can change product behavior without changing a screen. We therefore version the model, preprocessing specification, label mapping, and runtime compatibility together.

If models are downloaded after installation, we plan for interrupted transfers, storage shortages, integrity verification, authenticated distribution, and applicable platform rules. We verify artifacts against a trusted manifest before activation and retain a known-good version when feasible. A downloaded file should never become active merely because it has the expected filename.

Our evaluation matrix includes representative device tiers, operating-system versions, languages where relevant, and realistic capture conditions. We test poor lighting, noisy environments, unusual device placement, and missing sensor samples when those conditions matter to the task.

We roll out gradually and monitor operational signals such as initialization failures, inference duration, cancellations, and fallback frequency without collecting raw personal inputs by default. Rollback is part of the launch plan, not a feature we add after an update goes wrong.

A short pre-launch checklist

Before calling an on-device AI feature ready, we check that:

• The task has measurable acceptance criteria and a non-AI baseline.
• Supported devices pass sustained latency, memory, battery, and thermal tests.
• Flutter stays responsive, with bounded queues and cancellable work.
• Preprocessing matches training and is covered by automated tests.
• Permissions, retention, telemetry, and cloud fallback match our privacy claims.
• Uncertain results and unavailable inference have usable alternatives.
• Model updates are verified, versioned, observable, and reversible.

On-device AI works best when we treat intelligence as one component of a dependable product. The model matters, but so do sensor quality, interface decisions, platform constraints, and the moments when inference cannot help. If we are exploring that balance together, we can start with a focused prototype on the hardware people will actually use. Start a project with Kordspace, or explore our related mobile, AI, hardware, and design work through our services pages.