Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Holistic Evaluation of Language Models


Nov 16, 2022
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda

Add code

* Authored by the Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Project page: https://crfm.stanford.edu/helm/v1.0 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Prompts


Oct 14, 2021
Benjamin Newman, Prafulla Kumar Choubey, Nazneen Rajani

Add code

* 15 pages, 6 figures, 4 tables 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Refining Targeted Syntactic Evaluation of Language Models


Apr 19, 2021
Benjamin Newman, Kai-Siang Ang, Julia Gong, John Hewitt

Add code

* 14 pages, 5 figures, 3 tables. To appear at NAACL 2021 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Optimal Assistance for Object-Rearrangement Tasks in Augmented Reality


Oct 14, 2020
Benjamin Newman, Kevin Carlberg, Ruta Desai

Add code

* 19 pages including supplementary. Under review for ACM IUI 2021 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

The EOS Decision and Length Extrapolation


Oct 14, 2020
Benjamin Newman, John Hewitt, Percy Liang, Christopher D. Manning

Add code

* 16 page, 7 Figures, 9 Tables, Blackbox NLP Workshop at EMNLP 2020 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

Communication-based Evaluation for Natural Language Generation


Oct 11, 2019
Benjamin Newman, Reuben Cohn-Gordon, Christopher Potts

Add code

* 11 pages, 2 figures, SCiL, camera-ready - clarified certain points, updated acknowledgements 

   Access Paper or Ask Questions

  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email