Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

GPT-NeoX-20B: An Open-Source Autoregressive Language Model



Sid Black , Stella Biderman , Eric Hallahan , Quentin Anthony , Leo Gao , Laurence Golding , Horace He , Connor Leahy , Kyle McDonell , Jason Phang , Michael Pieler , USVSN Sai Prashanth , Shivanshu Purohit , Laria Reynolds , Jonathan Tow , Ben Wang , Samuel Weinbach

* To appear in the Proceedings of the ACL Workshop on Challenges & Perspectives in Creating Large Language Models 

   Access Paper or Ask Questions

Datasheet for the Pile



Stella Biderman , Kieran Bicheno , Leo Gao

* Accompanies "The Pile: An 800GB Dataset of Diverse Text for Language Modeling" arXiv:2101.00027 

   Access Paper or Ask Questions

Multitask Prompted Training Enables Zero-Shot Task Generalization



Victor Sanh , Albert Webson , Colin Raffel , Stephen H. Bach , Lintang Sutawika , Zaid Alyafeai , Antoine Chaffin , Arnaud Stiegler , Teven Le Scao , Arun Raja , Manan Dey , M Saiful Bari , Canwen Xu , Urmish Thakker , Shanya Sharma Sharma , Eliza Szczechla , Taewoon Kim , Gunjan Chhablani , Nihal Nayak , Debajyoti Datta , Jonathan Chang , Mike Tian-Jian Jiang , Han Wang , Matteo Manica , Sheng Shen , Zheng Xin Yong , Harshit Pandey , Rachel Bawden , Thomas Wang , Trishala Neeraj , Jos Rozen , Abheesht Sharma , Andrea Santilli , Thibault Fevry , Jason Alan Fries , Ryan Teehan , Stella Biderman , Leo Gao , Tali Bers , Thomas Wolf , Alexander M. Rush

* https://github.com/bigscience-workshop/promptsource/ 

   Access Paper or Ask Questions

Cut the CARP: Fishing for zero-shot story evaluation



Shahbuland Matiana , JR Smith , Ryan Teehan , Louis Castricato , Stella Biderman , Leo Gao , Spencer Frazier

* 9 pages, 4 figures 

   Access Paper or Ask Questions

An Empirical Exploration in Quality Filtering of Text Data



Leo Gao


   Access Paper or Ask Questions

The Pile: An 800GB Dataset of Diverse Text for Language Modeling



Leo Gao , Stella Biderman , Sid Black , Laurence Golding , Travis Hoppe , Charles Foster , Jason Phang , Horace He , Anish Thite , Noa Nabeshima , Shawn Presser , Connor Leahy


   Access Paper or Ask Questions

Collaborative Storytelling with Large-scale Neural Language Models



Eric Nichols , Leo Gao , Randy Gomez

* To appear in Proceedings of the 13th Annual ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG 2020) 

   Access Paper or Ask Questions