Is memory the moat? | Wafer

Wafer benchmarked Kimi K3 inference on AMD MI355X and found better performance per dollar than NVIDIA B200 and B300 setups.

Kimi K3 is so large it cannot fit on a single 8-GPU B200 node, making MI355X’s 288GB HBM a key advantage.

The MI355X delivered about 952 tokens per second per node, highlighting how memory capacity is becoming a competitive edge for frontier open-source model serving.

The report also notes that AMD still faces engineering challenges around ROCm, kernel maturity, and broader software support.