Learning Sparse Neural Networks Through L₀ Regularization

Researchers introduced a differentiable L0 regularization method for sparse neural networks.

The approach uses non-negative stochastic gates and a hard concrete distribution to push weights to exact zero.

This makes it possible to optimize sparsity with gradient descent instead of relying on post-training pruning.

The method can reduce computation and may improve generalization through model compression.