Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Daniel Soudry

Daniel Soudry

Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling


Jun 14, 2021
Niv Giladi, Zvika Ben-Haim, Sella Nevo, Yossi Matias, Daniel Soudry

Figure 1 for Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling
Figure 2 for Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling
Figure 3 for Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling
Figure 4 for Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling

  Access Paper or Ask Questions

Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks


Feb 25, 2021
Matan Haroush, Tzivel Frostig, Ruth Heller, Daniel Soudry

Figure 1 for Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks
Figure 2 for Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks
Figure 3 for Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks
Figure 4 for Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks

  Access Paper or Ask Questions

On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent


Feb 19, 2021
Shahar Azulay, Edward Moroshko, Mor Shpigel Nacson, Blake Woodworth, Nathan Srebro, Amir Globerson, Daniel Soudry

Figure 1 for On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent
Figure 2 for On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent
* 33 pages, 2 figures 
  Access Paper or Ask Questions

Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks


Feb 16, 2021
Itay Hubara, Brian Chmiel, Moshe Island, Ron Banner, Seffi Naor, Daniel Soudry

Figure 1 for Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
Figure 2 for Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
Figure 3 for Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
Figure 4 for Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks

  Access Paper or Ask Questions

Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates


Oct 01, 2020
Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry

Figure 1 for Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
Figure 2 for Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
Figure 3 for Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
Figure 4 for Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates
* The arXiv paper "Task Agnostic Continual Learning Using Online Variational Bayes" is a preliminary pre-print of this paper. The main differences between the versions are: 1. We develop new algorithmic framework (FOO-VB). 2. We add multivariate Gaussian and matrix variate Gaussian versions of the algorithm. 3. We demonstrate the new algorithm performance in task agnostic scenarios 
  Access Paper or Ask Questions

Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy


Jul 13, 2020
Edward Moroshko, Suriya Gunasekar, Blake Woodworth, Jason D. Lee, Nathan Srebro, Daniel Soudry

Figure 1 for Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
Figure 2 for Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
Figure 3 for Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy
Figure 4 for Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy

  Access Paper or Ask Questions

Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?


Jul 02, 2020
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry

Figure 1 for Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
Figure 2 for Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
Figure 3 for Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
Figure 4 for Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
* ICML 2020 
  Access Paper or Ask Questions

Neural gradients are lognormally distributed: understanding sparse and quantized training


Jun 17, 2020
Brian Chmiel, Liad Ben-Uri, Moran Shkolnik, Elad Hoffer, Ron Banner, Daniel Soudry

Figure 1 for Neural gradients are lognormally distributed: understanding sparse and quantized training
Figure 2 for Neural gradients are lognormally distributed: understanding sparse and quantized training
Figure 3 for Neural gradients are lognormally distributed: understanding sparse and quantized training
Figure 4 for Neural gradients are lognormally distributed: understanding sparse and quantized training
* Fix references typos 
  Access Paper or Ask Questions

Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming


Jun 14, 2020
Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry

Figure 1 for Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
Figure 2 for Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
Figure 3 for Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
Figure 4 for Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming

  Access Paper or Ask Questions

Kernel and Rich Regimes in Overparametrized Models


Feb 24, 2020
Blake Woodworth, Suriya Gunasekar, Jason D. Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, Nathan Srebro

Figure 1 for Kernel and Rich Regimes in Overparametrized Models
Figure 2 for Kernel and Rich Regimes in Overparametrized Models
Figure 3 for Kernel and Rich Regimes in Overparametrized Models
Figure 4 for Kernel and Rich Regimes in Overparametrized Models
* This updates and significantly extends a previous article (arXiv:1906.05827), Sections 6 and 7.1 are the most major additions. 30 pages. arXiv admin note: text overlap with arXiv:1906.05827 
  Access Paper or Ask Questions

MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks


Dec 29, 2019
Tzofnat Greenberg Toledo, Ben Perach, Daniel Soudry, Shahar Kvatinsky

Figure 1 for MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks
Figure 2 for MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks
Figure 3 for MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks
Figure 4 for MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks

  Access Paper or Ask Questions

Is Feature Diversity Necessary in Neural Network Initialization?


Dec 12, 2019
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry

Figure 1 for Is Feature Diversity Necessary in Neural Network Initialization?
Figure 2 for Is Feature Diversity Necessary in Neural Network Initialization?
Figure 3 for Is Feature Diversity Necessary in Neural Network Initialization?
Figure 4 for Is Feature Diversity Necessary in Neural Network Initialization?
* 4 + 1 pages. Workshop paper 
  Access Paper or Ask Questions

The Knowledge Within: Methods for Data-Free Model Compression


Dec 03, 2019
Matan Haroush, Itay Hubara, Elad Hoffer, Daniel Soudry

Figure 1 for The Knowledge Within: Methods for Data-Free Model Compression
Figure 2 for The Knowledge Within: Methods for Data-Free Model Compression
Figure 3 for The Knowledge Within: Methods for Data-Free Model Compression
Figure 4 for The Knowledge Within: Methods for Data-Free Model Compression

  Access Paper or Ask Questions

A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case


Oct 03, 2019
Greg Ongie, Rebecca Willett, Daniel Soudry, Nathan Srebro

Figure 1 for A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case
Figure 2 for A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case

  Access Paper or Ask Questions

At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?


Sep 26, 2019
Niv Giladi, Mor Shpigel Nacson, Elad Hoffer, Daniel Soudry

Figure 1 for At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?
Figure 2 for At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?
Figure 3 for At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?
Figure 4 for At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?

  Access Paper or Ask Questions

Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency


Aug 12, 2019
Elad Hoffer, Berry Weinstein, Itay Hubara, Tal Ben-Nun, Torsten Hoefler, Daniel Soudry

Figure 1 for Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency
Figure 2 for Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency
Figure 3 for Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency
Figure 4 for Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency

  Access Paper or Ask Questions

Kernel and Deep Regimes in Overparametrized Models


Jun 13, 2019
Blake Woodworth, Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

Figure 1 for Kernel and Deep Regimes in Overparametrized Models
Figure 2 for Kernel and Deep Regimes in Overparametrized Models
Figure 3 for Kernel and Deep Regimes in Overparametrized Models
Figure 4 for Kernel and Deep Regimes in Overparametrized Models
* 16 pages 
  Access Paper or Ask Questions

A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off


Jun 03, 2019
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry

Figure 1 for A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off
Figure 2 for A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off
Figure 3 for A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off
Figure 4 for A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off

  Access Paper or Ask Questions

Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models


May 17, 2019
Mor Shpigel Nacson, Suriya Gunasekar, Jason D. Lee, Nathan Srebro, Daniel Soudry

* ICML Camera ready version 
  Access Paper or Ask Questions

How do infinite width bounded norm networks look in function space?


Feb 13, 2019
Pedro Savarese, Itay Evron, Daniel Soudry, Nathan Srebro

Figure 1 for How do infinite width bounded norm networks look in function space?
Figure 2 for How do infinite width bounded norm networks look in function space?

  Access Paper or Ask Questions

Augment your batch: better training with larger batches


Jan 27, 2019
Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, Daniel Soudry

Figure 1 for Augment your batch: better training with larger batches
Figure 2 for Augment your batch: better training with larger batches
Figure 3 for Augment your batch: better training with larger batches
Figure 4 for Augment your batch: better training with larger batches

  Access Paper or Ask Questions

The Global Optimization Geometry of Shallow Linear Neural Networks


Nov 05, 2018
Zhihui Zhu, Daniel Soudry, Yonina C. Eldar, Michael B. Wakin

Figure 1 for The Global Optimization Geometry of Shallow Linear Neural Networks
Figure 2 for The Global Optimization Geometry of Shallow Linear Neural Networks
Figure 3 for The Global Optimization Geometry of Shallow Linear Neural Networks
Figure 4 for The Global Optimization Geometry of Shallow Linear Neural Networks

  Access Paper or Ask Questions

Characterizing Implicit Bias in Terms of Optimization Geometry


Oct 22, 2018
Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

Figure 1 for Characterizing Implicit Bias in Terms of Optimization Geometry
* (1) A bug in the proof of implicit bias for matrix factorization was fixed. v2 gives a characterization of the asymptotic bias of the factor matrices, while v1 made a stronger claim on the limit direction of the unfactored matrix. (2) v2 also includes new results on implicit bias of mirror descent with realizable affine constraints 
  Access Paper or Ask Questions

ACIQ: Analytical Clipping for Integer Quantization of neural networks


Oct 02, 2018
Ron Banner, Yury Nahshan, Elad Hoffer, Daniel Soudry

Figure 1 for ACIQ: Analytical Clipping for Integer Quantization of neural networks
Figure 2 for ACIQ: Analytical Clipping for Integer Quantization of neural networks
Figure 3 for ACIQ: Analytical Clipping for Integer Quantization of neural networks
Figure 4 for ACIQ: Analytical Clipping for Integer Quantization of neural networks

  Access Paper or Ask Questions

On the Blindspots of Convolutional Networks


Jul 08, 2018
Elad Hoffer, Shai Fine, Daniel Soudry

Figure 1 for On the Blindspots of Convolutional Networks
Figure 2 for On the Blindspots of Convolutional Networks
Figure 3 for On the Blindspots of Convolutional Networks
Figure 4 for On the Blindspots of Convolutional Networks

  Access Paper or Ask Questions

Scalable Methods for 8-bit Training of Neural Networks


Jun 17, 2018
Ron Banner, Itay Hubara, Elad Hoffer, Daniel Soudry

Figure 1 for Scalable Methods for 8-bit Training of Neural Networks
Figure 2 for Scalable Methods for 8-bit Training of Neural Networks
Figure 3 for Scalable Methods for 8-bit Training of Neural Networks

  Access Paper or Ask Questions

Convergence of Gradient Descent on Separable Data


Jun 12, 2018
Mor Shpigel Nacson, Jason Lee, Suriya Gunasekar, Pedro H. P. Savarese, Nathan Srebro, Daniel Soudry

Figure 1 for Convergence of Gradient Descent on Separable Data
Figure 2 for Convergence of Gradient Descent on Separable Data
Figure 3 for Convergence of Gradient Descent on Separable Data
Figure 4 for Convergence of Gradient Descent on Separable Data
* Added empirical results of experiments on deep networks (Appendix E). In addition, minor typos and phrasing mistakes were fixed 
  Access Paper or Ask Questions

Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate


Jun 05, 2018
Mor Shpigel Nacson, Nathan Srebro, Daniel Soudry

Figure 1 for Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate
Figure 2 for Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate
Figure 3 for Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate
* 7 pages in main paper, 10 pages of proofs in appendix, 2 figures, 1 table 
  Access Paper or Ask Questions

Implicit Bias of Gradient Descent on Linear Convolutional Networks


Jun 01, 2018
Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

Figure 1 for Implicit Bias of Gradient Descent on Linear Convolutional Networks

  Access Paper or Ask Questions