Sakana AI Proposes DiffusionBlocks: a Block-wise Training Framework That Converts Residual Networks into Independently Trainable Denoising Modules

Sakana AI and the University of Tokyo introduced DiffusionBlocks, a block-wise training framework for deep neural networks.

It turns residual blocks into independently trainable denoising modules based on diffusion ODEs, with each block trained on its own noise range.

By updating only one block per iteration and conditioning on noise, the method reduces memory use roughly in proportion to the number of blocks.

The authors say DiffusionBlocks preserves performance across architectures, including residual and transformer-based models, and outperforms prior greedy and Forward-Forward style methods.