How TetraMem uses analogue memory to run AI calculations
Illustrative image: panumas nikhomkhai on Pexels
TetraMem has taped out its 12 nm MLX300 AI system on chip, designed to hold up to 64 million INT4 model parameters on chip without external DRAM for those weights. The company is targeting 50–80 TOPS/W; measured silicon results will be needed to establish how closely complete workloads approach that target.
Its analogue architecture uses the stored weights in the calculation itself. Instead of repeatedly fetching them into separate arithmetic units, a resistive memory array forms weighted sums through the currents flowing in its cells.
Where the multiplication happens
In a simplified crossbar, each intersection stores a programmable conductance. Apply voltages along the rows and each cell produces a current proportional to its conductance and the applied voltage. The currents add at the columns, giving Iⱼ = ΣGᵢⱼVᵢ. The electrical network has performed the weighted sum.
Neural networks contain many operations that can be expressed in that form, so putting the weights into conductances can remove repeated transfers between memory and an arithmetic unit. TetraMem describes this mechanism in its published account of the architecture.
Getting the calculation into and out of the array still consumes energy. Digital values may need conversion before they can be applied electrically, and column currents have to become results the next stage can use. Converter precision and speed then affect both the numerical accuracy and the energy spent on an operation. Making the memory cell efficient will only carry the system so far if those peripheral circuits consume much of the budget.
Intermediate activations also need somewhere to go. A network can contain layers that map neatly onto an array alongside operations that require digital processing, with data moving between the two. Keeping weights on chip reduces one large source of traffic without eliminating the storage and transfers around it.
A model has to fit the array as well as the memory
At four bits per parameter, 64 million weights amount to 256 million bits, or 32 MB in decimal units. That is the raw weight payload, before implementation overhead. It does not describe a pool of general purpose SRAM.
Nor does a parameter count below the limit guarantee an efficient implementation. Layer shapes determine how the work is divided across arrays, and supported operations determine how often execution leaves the analogue path. A model can fit in the available weight storage and still spend too much time moving partial results or waiting for another part of the chip.
TetraMem’s software description includes model compilation and a RISC-V environment. The compiler has to preserve the required accuracy through quantisation while arranging the work around those physical constraints. Dividing a large layer across arrays, for example, creates partial results that must be combined somewhere; changing precision alters both the stored values and the result that can be expected from them.
Those decisions make the software part of the energy calculation. A successful implementation needs the array, converters and execution schedule to suit the model, with enough numerical accuracy left at the end to perform the intended task.
