Alert button
Picture for Margaret Li

Margaret Li

Alert button

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

Jan 19, 2024
Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah A. Smith, Luke Zettlemoyer

Viaarxiv icon

In-Context Pretraining: Language Modeling Beyond Document Boundaries

Oct 20, 2023
Weijia Shi, Sewon Min, Maria Lomeli, Chunting Zhou, Margaret Li, Xi Victoria Lin, Noah A. Smith, Luke Zettlemoyer, Scott Yih, Mike Lewis

Figure 1 for In-Context Pretraining: Language Modeling Beyond Document Boundaries
Figure 2 for In-Context Pretraining: Language Modeling Beyond Document Boundaries
Figure 3 for In-Context Pretraining: Language Modeling Beyond Document Boundaries
Figure 4 for In-Context Pretraining: Language Modeling Beyond Document Boundaries
Viaarxiv icon

Scaling Expert Language Models with Unsupervised Domain Discovery

Mar 24, 2023
Suchin Gururangan, Margaret Li, Mike Lewis, Weijia Shi, Tim Althoff, Noah A. Smith, Luke Zettlemoyer

Figure 1 for Scaling Expert Language Models with Unsupervised Domain Discovery
Figure 2 for Scaling Expert Language Models with Unsupervised Domain Discovery
Figure 3 for Scaling Expert Language Models with Unsupervised Domain Discovery
Figure 4 for Scaling Expert Language Models with Unsupervised Domain Discovery
Viaarxiv icon

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models

Aug 05, 2022
Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer

Figure 1 for Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
Figure 2 for Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
Figure 3 for Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
Figure 4 for Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models
Viaarxiv icon

Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers

Jul 26, 2021
Danielle Rothermel, Margaret Li, Tim Rocktäschel, Jakob Foerster

Figure 1 for Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers
Figure 2 for Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers
Figure 3 for Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers
Figure 4 for Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers
Viaarxiv icon

Recipes for Safety in Open-domain Chatbots

Oct 22, 2020
Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan

Figure 1 for Recipes for Safety in Open-domain Chatbots
Figure 2 for Recipes for Safety in Open-domain Chatbots
Figure 3 for Recipes for Safety in Open-domain Chatbots
Figure 4 for Recipes for Safety in Open-domain Chatbots
Viaarxiv icon

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds

Oct 01, 2020
Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, Jason Weston

Figure 1 for How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
Figure 2 for How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
Figure 3 for How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
Figure 4 for How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds
Viaarxiv icon

Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions

Jul 13, 2020
Stephen Roller, Y-Lan Boureau, Jason Weston, Antoine Bordes, Emily Dinan, Angela Fan, David Gunning, Da Ju, Margaret Li, Spencer Poff, Pratik Ringshia, Kurt Shuster, Eric Michael Smith, Arthur Szlam, Jack Urbanek, Mary Williamson

Viaarxiv icon