On-Device vs Cloud: What Actually Runs on Your Phone When You Use AI

When you use AI on your phone, some of it happens inside the device. Some of it happens on a server somewhere. Most users have no idea which is which, and the companies involved have not gone out of their way to make it obvious.

The distinction matters for three reasons: privacy, speed, and what happens when you have no internet connection. This article explains what runs locally on both iPhone and Android, what goes to the cloud, and why the boundary sits where it does.

Why the boundary exists at all

Running AI on a phone is a hardware problem. The large language models that power the most capable AI tools require enormous amounts of memory and processing power – far more than any current smartphone can provide. A model like GPT-4 runs on clusters of high-end server hardware. A phone, even a flagship one, has a fraction of that capacity, strict thermal limits, and a battery that cannot sustain sustained high-power computation.

The solution both Apple and Google have settled on is a tiered architecture. Simple, frequent tasks run locally on the device using a compact, optimised model. Tasks that exceed local capacity are escalated to cloud servers, where larger models handle the request and return a result. The phone decides which tier a given request requires.

What differs between the two companies is where exactly they draw the line, how they handle the cloud portion from a privacy standpoint, and how aggressively they are pushing that line toward the device.

Apple: on-device first, then Private Cloud Compute

Apple Intelligence runs a local model of approximately three billion parameters on the Neural Engine inside Apple Silicon – the A17 and A18 chips in recent iPhones, and the M-series chips in iPads and Macs. Apple has published performance figures for this model: it processes input at around 0.6 milliseconds per token on an iPhone 15 Pro and generates output at approximately 30 tokens per second. For a compact on-device model, that is fast enough to feel responsive in everyday use.

The tasks Apple routes through this local model cover a wide range of the features most users actually encounter. Writing tools – proofreading, rewriting, summarising – run locally. Notification summaries run locally. Basic image analysis runs locally. Predictive text, Live Text, and the majority of tool-calling operations within supported apps run locally. None of this data leaves the device.

When a request exceeds what the local model can handle, Apple Intelligence routes it to Private Cloud Compute. This is Apple’s own cloud infrastructure, running on Apple Silicon servers rather than conventional cloud hardware. The privacy architecture is specific: data is processed ephemerally, meaning it is not stored after the request is fulfilled. Apple states that it cannot access the content of requests sent to PCC. Independent security researchers can inspect the software running on PCC servers to verify these claims, and Apple publishes cryptographically signed binaries to support that verification.

Users can see which requests have been routed to PCC. In Settings, under Privacy and Security, the Apple Intelligence Report logs whether each request was handled on-device or sent to Private Cloud Compute, with options for the last 15 minutes or the last seven days.

The third tier is ChatGPT. When Siri determines that a request requires broader world knowledge that neither the local model nor PCC can adequately address, it can route to OpenAI’s ChatGPT. This is disabled by default. Users must opt in, and a confirmation prompt appears each time data would be shared with OpenAI. Apple’s PCC privacy guarantees do not apply to this tier – once data reaches OpenAI’s infrastructure, it is subject to OpenAI’s terms and privacy policy.

Android: Gemini Nano on the device, Gemini in the cloud

Google’s on-device model is Gemini Nano, which runs through a system service called AICore. AICore manages the model on behalf of the Android operating system, handling distribution, updates, and hardware acceleration. Apps access it through Google’s ML Kit GenAI APIs rather than directly.

Gemini Nano handles a defined set of tasks locally: summarising voice recordings, generating smart replies in messaging apps, live transcription, call summarisation, and basic image understanding. On Pixel devices, it runs on Google’s Tensor Processing Unit. On other flagship Android phones, it uses Qualcomm Snapdragon or MediaTek Dimensity neural processing hardware. The processing is event-driven rather than continuous – it activates when triggered by a specific user action and runs a brief inference, typically completing in 200 to 800 milliseconds.

Google has been extending Nano’s capabilities with each generation. Gemini Nano 4, announced at Google I/O 2025 and rolling out to flagship devices during 2026, is built on the Gemma 4 model architecture. Google states it runs up to four times faster than earlier Nano versions and reduces battery consumption for AI workloads by as much as 60 percent. It is multimodal, handling text, images, and audio in a single system, and supports more than 140 languages.

When a request goes beyond what Nano can handle – complex reasoning, multi-step research, long document synthesis – the Gemini app routes it to Google’s cloud infrastructure, where larger models including Gemini 2.5 Pro process the request. Unlike Apple’s PCC, this cloud processing takes place on standard cloud infrastructure. Google’s privacy policy governs what is retained and how it is used.

What runs locally: a practical comparison

On an iPhone running Apple Intelligence, the following happen entirely on the device: writing assistance in apps, notification summaries, basic photo editing through Clean Up, image classification, Live Text, predictive keyboard suggestions, and most Siri requests that draw on personal context such as calendar, contacts, and messages.

On an Android phone with Gemini Nano, the following happen locally: voice note summaries in the Recorder app, smart reply suggestions in Messages, call transcription, basic image descriptions for accessibility, and camera scene suggestions on supported devices.

Both platforms route to the cloud for anything requiring the largest model capabilities: extended reasoning, complex generative image creation, queries requiring broad world knowledge, and tasks that demand more memory than the local chip can address.

The hardware constraint

The line between on-device and cloud is not drawn by policy. It is drawn by physics. A smartphone’s neural processing unit has a fixed memory ceiling. A three-billion-parameter model, optimised and quantised for mobile, is roughly what fits within that ceiling while remaining responsive and power-efficient. Models ten or twenty times that size simply cannot run on current mobile hardware within acceptable power and thermal limits.

That ceiling rises with every chip generation. Apple’s Neural Engine and Google’s Tensor Processing Unit both improve with each annual release, as do the Qualcomm and MediaTek chips in Android flagships. Tasks that required cloud inference in 2023 run locally in 2025. The trajectory is consistent: more capability migrating to the device over time, with the cloud handling whatever remains at the frontier of model size.

What it means for privacy

Anything processed on-device never leaves the phone. That is the strongest privacy guarantee available, and it applies to both platforms for the tasks listed above.

For cloud processing, the two companies have taken different approaches. Apple has built dedicated infrastructure with verifiable privacy claims and no data retention after processing. Google routes through its general cloud infrastructure under standard privacy policies. Neither approach is inherently wrong, but they represent different positions on the trade-off between operational convenience and architectural privacy guarantees.

The ChatGPT integration in Apple Intelligence is a distinct case: Apple’s privacy architecture does not extend to OpenAI’s servers, and it is opt-in for precisely that reason.

The direction of travel

Both companies are pushing more AI onto the device. Google’s Gemini Nano 4 is a significant step in that direction. Apple’s annual chip improvements consistently expand what the local model can handle. The competitive pressure is straightforward: on-device AI is faster, works offline, and carries stronger privacy claims. Those are advantages worth pursuing.

Cloud AI is not going away. The most powerful models will remain cloud-based for the foreseeable future. But the question of what your phone sends to a server when you use AI features has a more reassuring answer in 2026 than it did two years ago, and the answer is improving every year.