Google’s Coral NPU joins the dots between an AI model and the silicon
Illustrative image: Sahand Babali on Unsplash
An AI accelerator can have an impressive arithmetic unit and still spend too much time waiting for data, or send awkward parts of a model back to the host CPU. Getting a neural network onto a chip involves decisions about memory, supported operations and the software that divides up the work.
Google’s latest Coral NPU release addresses those decisions together. The company has completed its 32-bit architecture with a matrix execution unit and released the foundation of an open compiler based on IREE and MLIR. Synaptics is bringing the design into new processors and development hardware, with Coralboard SL now generally available through DigiKey.
Keeping the arithmetic close to its data
The architecture combines RISC-V scalar control, vector processing and a dedicated matrix engine. Google describes an outer-product arrangement that reuses weights and activations in local registers rather than repeatedly fetching them for each piece of a matrix operation.
That reuse has an energy consequence as well as a throughput consequence. A multiply-accumulate operation needs operands, and moving those operands through a memory hierarchy is part of the work. Keeping a useful block of data near the arithmetic unit can reduce the transfers required to complete the next block of calculations.
Google describes configurations up to 1 TOPS and an average inference-power target around 10mW. Those describe the NPU architecture and its intended operating envelope. They are not a measurement of the total power drawn by a Coralboard, which also contains processor cores, memory, interfaces and supporting circuitry.
The compiler is the other half of the arrangement. It takes models from frameworks including JAX, PyTorch and TensorFlow, partitions work between the host and the accelerator, and arranges operations and memory use for the hardware. Publishing that toolchain makes the mapping available for inspection and development, rather than leaving it entirely inside a vendor executable.
Two boards with different jobs
Coralboard SL and the newly introduced Coralboard SR should not be treated as interchangeable versions of the same processor. The SL board uses Synaptics’ SL2619, with two Cortex-A55 cores, a Cortex-M52 and a 1 TOPS Torq/Coral NPU, alongside 2GB of DDR4.
The complete architecture release and each commercial implementation represent different stages of the design. Google identifies SR200’s second-generation Coral NPU as incorporating vector-core support; it does not say that every newly released architectural feature is already present in that silicon.
The SR board is based on the Astra SR250, pairing Cortex-M85 and Cortex-M52 processing with a 400 GOPS NPU and memory integrated into the processor platform. Synaptics lists up to 8MB of SRAM and 256MB of custom DRAM for that board configuration. Its camera, display and audio interfaces target a different mix of always-on sensing and multimodal work from the Linux-capable SL platform.
The wider SR200 family provides three pin-compatible performance tiers: SR210 at 200 GOPS, SR230 at 300 GOPS and SR250 at 400 GOPS. The common software environment is intended to let a design move between those tiers without beginning the application again.
Peak operation counts alone do not establish which platform will run a particular model faster or use less energy. Memory capacity, precision, operator support and the parts of the application running outside the NPU all affect the result. The two board architectures make those differences visible before a custom design reaches a PCB.
The SL board also provides CSI and DSI connections for cameras and displays, Qwiic and mikroBUS expansion, and an M.2 route to Wi-Fi and Bluetooth connectivity. Synaptics lists the wireless connectivity of the SR board separately, alongside support for multiple cameras, displays and digital microphones.
