Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Jacob Steinhardt

Measuring Coding Challenge Competence With APPS


May 27, 2021
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, Jacob Steinhardt

* Code and the APPS dataset is available at https://github.com/hendrycks/apps 

  Access Paper or Ask Questions

Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level


May 13, 2021
Ruiqi Zhong, Dhruba Ghosh, Dan Klein, Jacob Steinhardt

* ACL 2021 Findings. Code and data: https://github.com/ruiqi-zhong/acl2021-instance-level 

  Access Paper or Ask Questions

Agnostic learning with unknown utilities


Apr 17, 2021
Kush Bhatia, Peter L. Bartlett, Anca D. Dragan, Jacob Steinhardt

* 30 pages; published as a conference paper at ITCS 2021 

  Access Paper or Ask Questions

Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition


Mar 17, 2021
Yaodong Yu, Zitong Yang, Edgar Dobriban, Jacob Steinhardt, Yi Ma


  Access Paper or Ask Questions

Approximating How Single Head Attention Learns


Mar 13, 2021
Charlie Snell, Ruiqi Zhong, Dan Klein, Jacob Steinhardt


  Access Paper or Ask Questions

Limitations of Post-Hoc Feature Alignment for Robustness


Mar 10, 2021
Collin Burns, Jacob Steinhardt

* Accepted to CVPR 2021 

  Access Paper or Ask Questions

Measuring Mathematical Problem Solving With the MATH Dataset


Mar 05, 2021
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, Jacob Steinhardt

* Code and the MATH dataset is available at https://github.com/hendrycks/math/ 

  Access Paper or Ask Questions

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming


Nov 03, 2020
Sumanth Dathathri, Krishnamurthy Dvijotham, Alexey Kurakin, Aditi Raghunathan, Jonathan Uesato, Rudy Bunel, Shreya Shankar, Jacob Steinhardt, Ian Goodfellow, Percy Liang, Pushmeet Kohli


  Access Paper or Ask Questions

Measuring Massive Multitask Language Understanding


Sep 21, 2020
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, Jacob Steinhardt

* The test and code is available at https://github.com/hendrycks/test 

  Access Paper or Ask Questions

Aligning AI With Shared Human Values


Aug 05, 2020
Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, Jacob Steinhardt


  Access Paper or Ask Questions

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization


Jun 29, 2020
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, Justin Gilmer

* Datasets, code, and models available at https://github.com/hendrycks/imagenet-r 

  Access Paper or Ask Questions

Robust estimation via generalized quasi-gradients


May 28, 2020
Banghua Zhu, Jiantao Jiao, Jacob Steinhardt


  Access Paper or Ask Questions

Identifying Statistical Bias in Dataset Replication


May 19, 2020
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Jacob Steinhardt, Aleksander Madry


  Access Paper or Ask Questions

Rethinking Bias-Variance Trade-off for Generalization of Neural Networks


Mar 21, 2020
Zitong Yang, Yaodong Yu, Chong You, Jacob Steinhardt, Yi Ma


  Access Paper or Ask Questions

When does the Tukey median work?


Jan 21, 2020
Banghua Zhu, Jiantao Jiao, Jacob Steinhardt


  Access Paper or Ask Questions

A Benchmark for Anomaly Segmentation


Nov 25, 2019
Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, Dawn Song

* StreetHazards dataset and code are available at https://github.com/hendrycks/anomaly-seg 

  Access Paper or Ask Questions

Generalized Resilience and Robust Statistics


Sep 19, 2019
Banghua Zhu, Jiantao Jiao, Jacob Steinhardt


  Access Paper or Ask Questions

Testing Robustness Against Unforeseen Adversaries


Aug 21, 2019
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, Jacob Steinhardt


  Access Paper or Ask Questions

Natural Adversarial Examples


Jul 18, 2019
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, Dawn Song

* Dataset and code available at https://github.com/hendrycks/natural-adv-examples 

  Access Paper or Ask Questions

Transfer of Adversarial Robustness Between Perturbation Types


May 03, 2019
Daniel Kang, Yi Sun, Tom Brown, Dan Hendrycks, Jacob Steinhardt

* 11 pages, 6 figures 

  Access Paper or Ask Questions

Semidefinite relaxations for certifying robustness to adversarial examples


Nov 02, 2018
Aditi Raghunathan, Jacob Steinhardt, Percy Liang

* To appear at NIPS 2018 

  Access Paper or Ask Questions

Stronger Data Poisoning Attacks Break Data Sanitization Defenses


Nov 02, 2018
Pang Wei Koh, Jacob Steinhardt, Percy Liang


  Access Paper or Ask Questions

Does robustness imply tractability? A lower bound for planted clique in the semi-random model


Sep 04, 2018
Jacob Steinhardt

* Improved lower bound to give recovery probability tending to zero. Factored out and highlighted perturbed Bernoulli argument 

  Access Paper or Ask Questions

Troubling Trends in Machine Learning Scholarship


Jul 26, 2018
Zachary C. Lipton, Jacob Steinhardt

* Presented at ICML 2018: The Debates 

  Access Paper or Ask Questions

Sever: A Robust Meta-Algorithm for Stochastic Optimization


Mar 07, 2018
Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Jacob Steinhardt, Alistair Stewart


  Access Paper or Ask Questions