TinyML on Microcontrollers

TinyML is machine learning inference on microcontrollers - devices with kilobytes of RAM, no operating system, and power budgets measured in milliwatts. This is the extreme edge of the AI spectrum: the same neural network principles used in GPT-4, applied to chips that cost $1 and run on a AA battery for a year.

Why Run ML on a Microcontroller?

• Latency - inference in milliseconds, no cloud round-trip
• Privacy - "Hey Alexa" detection runs on the chip; audio never leaves the device until wake word is confirmed
• Power - a $1 microcontroller uses 10–100mW at full load vs a Raspberry Pi's 3–5W; enables battery-powered sensors that last years
• Cost - microcontrollers cost $0.50–$5; Raspberry Pi costs $35–$80
• Reliability - no Linux, no network stack, no cloud dependency; single-purpose, deterministic
• Scale - billions of IoT devices could run TinyML; no cloud costs at scale

What TinyML Can Do

• Keyword detection - "Hey Siri," "OK Google," "Alexa" all run on a dedicated microcontroller in the device; only after the wake word is detected does the more powerful processor activate
• Anomaly detection - industrial sensors detecting unusual vibration patterns in motors before failure (predictive maintenance)
• Gesture recognition - recognizing hand gestures from accelerometer data for wearables and AR controllers
• Image classification - camera-equipped microcontrollers classifying objects at the edge (crop disease detection, wildlife monitoring, quality inspection)
• Audio classification - detecting gunshots, glass breaking, or machinery faults from audio - without sending audio to a server
• Vital sign monitoring - detecting heart rate, respiratory rate, or fall events from wearable sensors

Hardware Constraints

Typical TinyML target: STM32F4 or Arduino Nano 33 BLE Sense
  RAM:    256KB–2MB
  Flash:  512KB–16MB
  CPU:    Cortex-M4 @ 168MHz (no FPU for some variants)
  Power:  3–30mA during inference
  Cost:   $1–10

Model budget:
  Model size: < 256KB flash (weights must fit in flash)
  Runtime RAM: < 64KB (activations + scratch buffers)
  Inference time: < 10ms for real-time tasks

TinyML Frameworks

Designing Models for TinyML

Standard neural network architectures do not fit on microcontrollers. TinyML requires purpose-designed architectures:

• MobileNet - depthwise separable convolutions replace standard convolutions, reducing computation by 8–9×
• EfficientNet-Lite - scaled MobileNet variants optimized for latency on edge hardware
• DS-CNN (Depthwise Separable CNN) - the standard for keyword detection (used in "Hey Siri" class models)
• Temporal Convolutional Networks (TCN) - efficient for time-series (accelerometer, sensor) classification

Key optimization techniques:

• Quantization - convert 32-bit float weights to 8-bit or even 4-bit integers. 4× smaller model, 4× faster inference on CPUs without floating-point hardware
• Pruning - zero out small weights; after training, remove them. Reduces model size by 50–90% with minimal accuracy loss
• Knowledge distillation - train a small model to mimic a large model's outputs; the small model learns from the large model's "soft" predictions

Edge Impulse - The Practical Path

Edge Impulse is the most accessible TinyML platform for practitioners who don't want to manually write C model deployment code:

• Collect data from sensors (accelerometer, microphone, camera) directly from the device
• Label data in the browser-based UI
• Train a model with a few clicks (or custom Keras/PyTorch)
• Automatically generate optimized C++ code for your target hardware
• Deploy via Arduino library, platformio, or direct flash
• Supports 60+ hardware platforms including Arduino, Raspberry Pi Pico, STM32, nRF52840, and more