Mobile AI - Apple & Google

The two dominant mobile platforms are taking different but converging approaches to on-device AI. Both are deploying small language models directly on consumer hardware - making AI inference private, fast, and available without an internet connection. The architectures they have built reveal how edge AI at billions-of-devices scale actually works.

Apple Intelligence

Launched in 2024–2025 across iPhone 15 Pro and later, iPad, and Mac. Apple Intelligence is a system-level AI deeply integrated with iOS, iPadOS, and macOS - not just a chatbot added on top.

Architecture

User Request (e.g., "Summarize my emails from today")
                     │
          ┌──────────▼──────────┐
          │ On-Device Classifier  │  ← Runs locally, always
          │ "Can this be done    │
          │  on-device?"         │
          └────┬──────────┬──────┘
               │          │
          Yes  │          │  No (too complex)
               ▼          ▼
        On-Device      Private Cloud Compute
        Model (~3B)    (Apple servers, custom
        Runs on ANE    Apple Silicon, encrypted,
        Zero network   ephemeral - no data retained)
        Pure privacy   ← Cryptographically verified

Key design decisions:

• On-device first: Summarization, rewriting, reply suggestions, image editing (Clean Up, Genmoji) - all run locally on the Apple Neural Engine.
• Private Cloud Compute: When a request needs more capability than the on-device model can provide, it routes to Apple's servers. Apple publishes the binary for independent security researchers to verify that the server code matches their stated privacy guarantees - an unusual level of transparency.
• Deep OS integration: The AI has "personal context" - it can see your emails, messages, calendar, contacts, and photos across apps. This is not possible for third-party AI apps on iOS.
• ChatGPT integration: For requests beyond Private Cloud Compute's capability, Apple routes to ChatGPT with explicit user permission - no silently sending data.

What Apple Intelligence Does

• Writing Tools: rewrite, proofread, summarize any text field across the OS
• Email and Message summaries
• Smart Reply suggestions with context from the full conversation
• Priority inbox: surfaces urgent messages
• Image generation: Genmoji, Image Playground (on-device diffusion)
• Photo editing: Clean Up (AI-powered remove tool, runs on ANE)
• Siri with screen context: Siri can see what is on your screen and take action

Google On-Device AI

Google's on-device AI strategy is built around Gemini Nano - a family of small models designed to run on Android phones without cloud access.

Gemini Nano

• Nano 1: 1.8B parameters - fits on older Pixel devices (Pixel 8)
• Nano 2: 3.25B parameters - for Pixel 8 Pro and later, Tensor G3+
• Nano-X: Enhanced version for flagships, multimodal (image + audio input)

Deployed capabilities on Pixel phones:

• Smart Reply in Gboard - suggests replies in any messaging app
• Magic Compose - rewrites messages in different tones (professional, casual)
• Recorder app: real-time transcription + summarization of recordings
• Live translation: translate conversations in real time, on-device, offline
• Pixel Screenshots: search your screenshots by content using on-device AI

MediaPipe - Google's On-Device ML Framework

MediaPipe is Google's cross-platform framework for on-device ML pipelines beyond language: pose detection, hand tracking, face mesh, object detection, image segmentation. It runs on Android, iOS, and web. Key capabilities:

• Hand landmark detection: 21 keypoints per hand at 30+ FPS on mobile
• Pose estimation: 33 body keypoints for fitness, AR, and accessibility
• Face detection and mesh: 478 face landmarks for AR filters, gaze estimation
• LLM inference: MediaPipe LLM Inference API lets Android/iOS developers run Gemma 2B, Phi-2, and other SLMs directly on device

Qualcomm - The Third Pillar

Qualcomm's Snapdragon 8 Gen 3 and later chips power most non-Apple premium Android phones. Their Hexagon NPU provides up to 75 TOPS and runs models from Google, Meta, Microsoft, and others. Qualcomm has partnered with Meta (Llama 3), Microsoft (Phi-3), and others to certify their models for on-device Snapdragon deployment - a quality tier they call "Snapdragon AI Ready."

Platform Comparison