Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation



Mark Diaz , Ian D. Kivlichan , Rachel Rosen , Dylan K. Baker , Razvan Amironesei , Vinodkumar Prabhakaran , Emily Denton

* 11 pages, Accepted at 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT). arXiv admin note: text overlap with arXiv:2112.04554 

   Access Paper or Ask Questions

Evaluation Gaps in Machine Learning Practice



Ben Hutchinson , Negar Rostamzadeh , Christina Greer , Katherine Heller , Vinodkumar Prabhakaran


   Access Paper or Ask Questions

PaLM: Scaling Language Modeling with Pathways



Aakanksha Chowdhery , Sharan Narang , Jacob Devlin , Maarten Bosma , Gaurav Mishra , Adam Roberts , Paul Barham , Hyung Won Chung , Charles Sutton , Sebastian Gehrmann , Parker Schuh , Kensen Shi , Sasha Tsvyashchenko , Joshua Maynez , Abhishek Rao , Parker Barnes , Yi Tay , Noam Shazeer , Vinodkumar Prabhakaran , Emily Reif , Nan Du , Ben Hutchinson , Reiner Pope , James Bradbury , Jacob Austin , Michael Isard , Guy Gur-Ari , Pengcheng Yin , Toju Duke , Anselm Levskaya , Sanjay Ghemawat , Sunipa Dev , Henryk Michalewski , Xavier Garcia , Vedant Misra , Kevin Robinson , Liam Fedus , Denny Zhou , Daphne Ippolito , David Luan , Hyeontaek Lim , Barret Zoph , Alexander Spiridonov , Ryan Sepassi , David Dohan , Shivani Agrawal , Mark Omernick , Andrew M. Dai , Thanumalayan Sankaranarayana Pillai , Marie Pellat , Aitor Lewkowycz , Erica Moreira , Rewon Child , Oleksandr Polozov , Katherine Lee , Zongwei Zhou , Xuezhi Wang , Brennan Saeta , Mark Diaz , Orhan Firat , Michele Catasta , Jason Wei , Kathy Meier-Hellstern , Douglas Eck , Jeff Dean , Slav Petrov , Noah Fiedel


   Access Paper or Ask Questions

LaMDA: Language Models for Dialog Applications



Romal Thoppilan , Daniel De Freitas , Jamie Hall , Noam Shazeer , Apoorv Kulshreshtha , Heng-Tze Cheng , Alicia Jin , Taylor Bos , Leslie Baker , Yu Du , YaGuang Li , Hongrae Lee , Huaixiu Steven Zheng , Amin Ghafouri , Marcelo Menegali , Yanping Huang , Maxim Krikun , Dmitry Lepikhin , James Qin , Dehao Chen , Yuanzhong Xu , Zhifeng Chen , Adam Roberts , Maarten Bosma , Vincent Zhao , Yanqi Zhou , Chung-Ching Chang , Igor Krivokon , Will Rusch , Marc Pickett , Pranesh Srinivasan , Laichee Man , Kathleen Meier-Hellstern , Meredith Ringel Morris , Tulsee Doshi , Renelito Delos Santos , Toju Duke , Johnny Soraker , Ben Zevenbergen , Vinodkumar Prabhakaran , Mark Diaz , Ben Hutchinson , Kristen Olson , Alejandra Molina , Erin Hoffman-John , Josh Lee , Lora Aroyo , Ravi Rajakumar , Alena Butryna , Matthew Lamm , Viktoriya Kuzmina , Joe Fenton , Aaron Cohen , Rachel Bernstein , Ray Kurzweil , Blaise Aguera-Arcas , Claire Cui , Marian Croak , Ed Chi , Quoc Le


   Access Paper or Ask Questions

Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation



Emily Denton , Mark Díaz , Ian Kivlichan , Vinodkumar Prabhakaran , Rachel Rosen


   Access Paper or Ask Questions

Thinking Beyond Distributions in Testing Machine Learned Models



Negar Rostamzadeh , Ben Hutchinson , Christina Greer , Vinodkumar Prabhakaran

* NeurIPS 2021 workshop on Distribution Shifts 
* Neural Information Processing System, NeurIPS 2021 workshop on Distribution Shifts 

   Access Paper or Ask Questions

Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations



Aida Mostafazadeh Davani , Mark Díaz , Vinodkumar Prabhakaran


   Access Paper or Ask Questions

On Releasing Annotator-Level Labels and Information in Datasets



Vinodkumar Prabhakaran , Aida Mostafazadeh Davani , Mark Díaz


   Access Paper or Ask Questions

1
2
3
>>