What femtoAI's open evaluation platform must prove

FemtoAI has opened its developer programme more widely following a private beta, providing access to an evaluation kit built around the company's Sparse Processing Unit.

The package includes compressed models, a compiler, a tool called Sparsity Studio and a developer forum. Participants can bring an existing model, use one supplied by femtoAI or build a new workload around the evaluation hardware.

The company claims that its architecture can reduce power consumption and memory requirements by a factor of ten without reducing model accuracy. These are femtoAI's figures and have not been independently established across a common public benchmark suite.

Named users include Legato, Marshall and NewSound, with initial applications concentrated in audio processing and sensor analysis.

Where sparsity can save energy

Neural networks often contain weights or activations that contribute little to the final result. Pruning, compression and sparsity techniques attempt to remove or avoid operations involving those values.

The potential saving is not confined to arithmetic. Moving weights and intermediate data between external memory and a processor can consume substantial energy. An architecture that keeps a smaller model close to the compute units and avoids unnecessary transfers may therefore improve both latency and power.

Real performance depends on the type of sparsity and whether the hardware can exploit it efficiently. A model can contain many zero values while still performing poorly if they occur in a pattern that the processor or compiler cannot skip.

Establishing the comparison

A credible evaluation needs a clearly defined baseline. A tenfold improvement against an unoptimised software implementation is not equivalent to the same improvement against a current microcontroller, DSP or neural-processing unit running an optimised model.

Power should also be measured at an appropriate boundary. Core or accelerator power does not include every contribution from memory, voltage conversion, sensors and supporting processors.

Accuracy has to be tested using the same dataset and acceptance criterion before and after conversion. Latency distribution, start-up time, memory footprint, sustained thermal behaviour and tool stability also influence whether the architecture is practical for production.

Opening the platform gives developers a route to collect this evidence on their own workloads. The strongest validation will come from repeatable measurements that state the model, compiler version, numerical precision, power boundary and comparison platform.