Alert button

Dynamic Model Pruning with Feedback

Jun 12, 2020
Tao Lin, Sebastian U. Stich, Luis Barba, Daniil Dmitriev, Martin Jaggi

Figure 1 for Dynamic Model Pruning with Feedback
Figure 2 for Dynamic Model Pruning with Feedback
Figure 3 for Dynamic Model Pruning with Feedback
Figure 4 for Dynamic Model Pruning with Feedback

Share this with someone who'll enjoy it:

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that generates a sparse trained model without additional overhead: by allowing (i) dynamic allocation of the sparsity pattern and (ii) incorporating feedback signal to reactivate prematurely pruned weights we obtain a performant sparse model in one single training pass (retraining is not needed, but can further improve the performance). We evaluate our method on CIFAR-10 and ImageNet, and show that the obtained sparse models can reach the state-of-the-art performance of dense models. Moreover, their performance surpasses that of models generated by all previously proposed pruning schemes.

* appearing at ICLR 2020  
View paper onarxiv iconopen_review iconOpenReview

Share this with someone who'll enjoy it: