Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Daniel Soudry

Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling


Jun 14, 2021
Niv Giladi, Zvika Ben-Haim, Sella Nevo, Yossi Matias, Daniel Soudry


  Access Paper or Ask Questions

Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks


Feb 25, 2021
Matan Haroush, Tzivel Frostig, Ruth Heller, Daniel Soudry


  Access Paper or Ask Questions

On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent


Feb 19, 2021
Shahar Azulay, Edward Moroshko, Mor Shpigel Nacson, Blake Woodworth, Nathan Srebro, Amir Globerson, Daniel Soudry

* 33 pages, 2 figures 

  Access Paper or Ask Questions

Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks


Feb 16, 2021
Itay Hubara, Brian Chmiel, Moshe Island, Ron Banner, Seffi Naor, Daniel Soudry


  Access Paper or Ask Questions

Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates


Oct 01, 2020
Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry

* The arXiv paper "Task Agnostic Continual Learning Using Online Variational Bayes" is a preliminary pre-print of this paper. The main differences between the versions are: 1. We develop new algorithmic framework (FOO-VB). 2. We add multivariate Gaussian and matrix variate Gaussian versions of the algorithm. 3. We demonstrate the new algorithm performance in task agnostic scenarios 

  Access Paper or Ask Questions

Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy


Jul 13, 2020
Edward Moroshko, Suriya Gunasekar, Blake Woodworth, Jason D. Lee, Nathan Srebro, Daniel Soudry


  Access Paper or Ask Questions

Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?


Jul 02, 2020
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry

* ICML 2020 

  Access Paper or Ask Questions

Neural gradients are lognormally distributed: understanding sparse and quantized training


Jun 17, 2020
Brian Chmiel, Liad Ben-Uri, Moran Shkolnik, Elad Hoffer, Ron Banner, Daniel Soudry

* Fix references typos 

  Access Paper or Ask Questions

Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming


Jun 14, 2020
Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry


  Access Paper or Ask Questions

Kernel and Rich Regimes in Overparametrized Models


Feb 24, 2020
Blake Woodworth, Suriya Gunasekar, Jason D. Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, Nathan Srebro

* This updates and significantly extends a previous article (arXiv:1906.05827), Sections 6 and 7.1 are the most major additions. 30 pages. arXiv admin note: text overlap with arXiv:1906.05827 

  Access Paper or Ask Questions

MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks


Dec 29, 2019
Tzofnat Greenberg Toledo, Ben Perach, Daniel Soudry, Shahar Kvatinsky


  Access Paper or Ask Questions

Is Feature Diversity Necessary in Neural Network Initialization?


Dec 12, 2019
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry

* 4 + 1 pages. Workshop paper 

  Access Paper or Ask Questions

The Knowledge Within: Methods for Data-Free Model Compression


Dec 03, 2019
Matan Haroush, Itay Hubara, Elad Hoffer, Daniel Soudry


  Access Paper or Ask Questions

A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case


Oct 03, 2019
Greg Ongie, Rebecca Willett, Daniel Soudry, Nathan Srebro


  Access Paper or Ask Questions

At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?


Sep 26, 2019
Niv Giladi, Mor Shpigel Nacson, Elad Hoffer, Daniel Soudry


  Access Paper or Ask Questions

Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency


Aug 12, 2019
Elad Hoffer, Berry Weinstein, Itay Hubara, Tal Ben-Nun, Torsten Hoefler, Daniel Soudry


  Access Paper or Ask Questions

Kernel and Deep Regimes in Overparametrized Models


Jun 13, 2019
Blake Woodworth, Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

* 16 pages 

  Access Paper or Ask Questions

A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off


Jun 03, 2019
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry


  Access Paper or Ask Questions

Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models


May 17, 2019
Mor Shpigel Nacson, Suriya Gunasekar, Jason D. Lee, Nathan Srebro, Daniel Soudry

* ICML Camera ready version 

  Access Paper or Ask Questions

How do infinite width bounded norm networks look in function space?


Feb 13, 2019
Pedro Savarese, Itay Evron, Daniel Soudry, Nathan Srebro


  Access Paper or Ask Questions

Augment your batch: better training with larger batches


Jan 27, 2019
Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, Daniel Soudry


  Access Paper or Ask Questions

The Global Optimization Geometry of Shallow Linear Neural Networks


Nov 05, 2018
Zhihui Zhu, Daniel Soudry, Yonina C. Eldar, Michael B. Wakin


  Access Paper or Ask Questions

Characterizing Implicit Bias in Terms of Optimization Geometry


Oct 22, 2018
Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

* (1) A bug in the proof of implicit bias for matrix factorization was fixed. v2 gives a characterization of the asymptotic bias of the factor matrices, while v1 made a stronger claim on the limit direction of the unfactored matrix. (2) v2 also includes new results on implicit bias of mirror descent with realizable affine constraints 

  Access Paper or Ask Questions

ACIQ: Analytical Clipping for Integer Quantization of neural networks


Oct 02, 2018
Ron Banner, Yury Nahshan, Elad Hoffer, Daniel Soudry


  Access Paper or Ask Questions

On the Blindspots of Convolutional Networks


Jul 08, 2018
Elad Hoffer, Shai Fine, Daniel Soudry


  Access Paper or Ask Questions

Scalable Methods for 8-bit Training of Neural Networks


Jun 17, 2018
Ron Banner, Itay Hubara, Elad Hoffer, Daniel Soudry


  Access Paper or Ask Questions

Convergence of Gradient Descent on Separable Data


Jun 12, 2018
Mor Shpigel Nacson, Jason Lee, Suriya Gunasekar, Pedro H. P. Savarese, Nathan Srebro, Daniel Soudry

* Added empirical results of experiments on deep networks (Appendix E). In addition, minor typos and phrasing mistakes were fixed 

  Access Paper or Ask Questions

Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate


Jun 05, 2018
Mor Shpigel Nacson, Nathan Srebro, Daniel Soudry

* 7 pages in main paper, 10 pages of proofs in appendix, 2 figures, 1 table 

  Access Paper or Ask Questions

Implicit Bias of Gradient Descent on Linear Convolutional Networks


Jun 01, 2018
Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro


  Access Paper or Ask Questions