Intel splits AI inference between its D270 GPU and SambaNova’s SN50

Illustrative image: Mahdi Bafande on Pexels
Intel and SambaNova are demonstrating an inference system at SC26 that assigns prompt processing to four Intel D270 GPUs and response generation to a SambaRack SN50. Xeon hosts coordinate the two accelerator pools, allowing their capacity to be adjusted separately as the balance between incoming prompts and generated output changes.
The division follows the different demands of prefill and decode. Prefill processes the prompt and builds the cache of key and value tensors needed during subsequent calculations, with long prompts offering substantial parallel work. Decode then generates the response a token at a time, repeatedly drawing on that stored state. In the demonstration, the cache travels between the two pools over Ethernet using remote direct memory access.
SambaNova describes the D270 used for prefill as a 350 W PCIe card with 160 GB of LPDDR5X and 1.5 TB/s of memory bandwidth. Four cards serve this show configuration, which runs GLM-5.3 across the two hardware platforms; the larger 80 GPU reference rack shown previously at OCP is a separate configuration.
For decode, the SN50 uses reconfigurable dataflow processing. SambaNova says it can execute a chain of operations without repeatedly writing intermediate values back to memory between kernels, reducing those transfers as the calculation proceeds. A SambaRack SN50 contains 16 processors and has typical inference power of about 20 kW, excluding the Intel side of the combined installation.
The software coordinates where work runs as well as where the cache resides. SambaNova describes vLLM operating on both sides, with Nvidia Dynamo directing requests towards a server that already holds the relevant context. Reusing that context avoids repeating prefill, while a new request incurs the transfer of its newly constructed cache before decode can begin.
The system operates in a facility using air cooling, although the partners have not published comparative latency or throughput results for the combined deployment. Intel’s separate D270 benchmarks, including tests with Qwen 3 30B A3B, were run on preproduction silicon and compare individual GPUs. Measurements of the complete arrangement will need to include the exchanges between prefill and decode as well as the processing within each pool.
Sources: Intel SC26 technical briefing and test notes; SambaNova’s platform demonstration and serving stack.



