Perception hardware
Sensors, signal processors, timing, calibration, and local preprocessing.
Layer 6 — AI Hardware · Chapter 6
Not all AI computation belongs in a data center. Many systems must interpret the world and act where data is produced: inside phones, vehicles, cameras, medical devices, factories, farms, homes, and robots. Edge AI combines specialized processors with sensors, local memory, communication, power management, and sometimes actuators.
Edge devices have limited power, cooling, memory, and physical space. Many operate from batteries. They may be exposed to vibration, heat, cold, dust, moisture, or years of unattended use. A model that works in a data center may need compression, quantization, pruning, distillation, or architectural changes before it can run locally.
| Constraint | Hardware response | Model response |
|---|---|---|
| Limited energy | Efficient accelerators, sleep modes, event-driven sensors | Smaller models, conditional computation, lower precision |
| Limited memory | On-chip memory, compact storage, shared memory systems | Compression, streaming, retrieval, shorter context |
| Real-time deadlines | Dedicated pipelines and deterministic scheduling | Bounded computation and fallback behavior |
| Harsh environment | Rugged packaging, redundancy, thermal control | Robustness to missing or degraded sensor data |
| Long product life | Serviceable modules and secure update mechanisms | Backward compatibility and continual validation |
Sensors convert physical phenomena into data. Cameras capture visible or infrared light. Microphones capture sound. Radar and lidar measure distance and motion. Inertial sensors measure acceleration and rotation. Force, pressure, temperature, chemical, biological, and location sensors provide additional forms of context.
Sensor quality determines what the model can know. Resolution, dynamic range, noise, calibration, placement, synchronization, and failure behavior all affect the reliability of perception. More sensors can improve coverage but also add cost, power use, data volume, and integration complexity.
Robust systems combine several sensing modes. A vehicle may use cameras for semantic detail, radar for distance and velocity, and inertial sensors for motion. A robot may combine vision, joint encoders, force feedback, and touch. Fusion can compensate for individual weaknesses, but it requires accurate timing, calibration, and models that understand uncertainty.
Robotics adds motors, drives, valves, grippers, wheels, legs, and other actuators. This changes the nature of AI risk. A mistaken text answer can be corrected; a mistaken physical action can damage property or injure a person. Hardware therefore needs limits, emergency stops, force control, redundancy, and safe states independent of the main AI model.
Sensors, signal processors, timing, calibration, and local preprocessing.
CPUs, accelerators, memory, storage, and real-time controllers.
Motors, drives, power electronics, mechanisms, and physical end effectors.
Independent monitors, limits, brakes, emergency systems, and fault detection.
Most systems will divide work rather than choose only cloud or edge. Devices can perform immediate perception and control, while cloud systems provide larger models, fleet learning, simulation, long-term memory, and updates. The architecture must decide which data leave the device, which decisions can wait, and how the system behaves when the connection fails.
In edge and robotics markets, industrial design, battery life, acoustics, weight, durability, maintenance, and cost may matter as much as benchmark performance. The winning system must fit into a human environment and survive repeated real-world use.
Embodied AI begins where prediction meets physics. Sensors define what can be perceived, and actuators define what errors can cost.