Alert button
Picture for Christopher Ré

Christopher Ré

Alert button

State-Free Inference of State-Space Models: The Transfer Function Approach

Add code
Bookmark button
Alert button
May 10, 2024
Rom N. Parnichkun, Stefano Massaroli, Alessandro Moro, Jimmy T. H. Smith, Ramin Hasani, Mathias Lechner, Qi An, Christopher Ré, Hajime Asama, Stefano Ermon, Taiji Suzuki, Atsushi Yamashita, Michael Poli

Viaarxiv icon

Mechanistic Design and Scaling of Hybrid Architectures

Add code
Bookmark button
Alert button
Mar 26, 2024
Michael Poli, Armin W Thomas, Eric Nguyen, Pragaash Ponnusamy, Björn Deiseroth, Kristian Kersting, Taiji Suzuki, Brian Hie, Stefano Ermon, Christopher Ré, Ce Zhang, Stefano Massaroli

Figure 1 for Mechanistic Design and Scaling of Hybrid Architectures
Figure 2 for Mechanistic Design and Scaling of Hybrid Architectures
Figure 3 for Mechanistic Design and Scaling of Hybrid Architectures
Figure 4 for Mechanistic Design and Scaling of Hybrid Architectures
Viaarxiv icon

Simple linear attention language models balance the recall-throughput tradeoff

Add code
Bookmark button
Alert button
Feb 28, 2024
Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, Dylan Zinsley, James Zou, Atri Rudra, Christopher Ré

Viaarxiv icon

Prospector Heads: Generalized Feature Attribution for Large Models & Data

Add code
Bookmark button
Alert button
Feb 18, 2024
Gautam Machiraju, Alexander Derry, Arjun Desai, Neel Guha, Amir-Hossein Karimi, James Zou, Russ Altman, Christopher Ré, Parag Mallick

Viaarxiv icon

Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT

Add code
Bookmark button
Alert button
Feb 14, 2024
Jon Saad-Falcon, Daniel Y. Fu, Simran Arora, Neel Guha, Christopher Ré

Figure 1 for Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT
Figure 2 for Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT
Figure 3 for Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT
Figure 4 for Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT
Viaarxiv icon

Hydragen: High-Throughput LLM Inference with Shared Prefixes

Add code
Bookmark button
Alert button
Feb 07, 2024
Jordan Juravsky, Bradley Brown, Ryan Ehrlich, Daniel Y. Fu, Christopher Ré, Azalia Mirhoseini

Viaarxiv icon

The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

Add code
Bookmark button
Alert button
Feb 06, 2024
Michael Zhang, Kush Bhatia, Hermann Kumbong, Christopher Ré

Viaarxiv icon

Zoology: Measuring and Improving Recall in Efficient Language Models

Add code
Bookmark button
Alert button
Dec 08, 2023
Simran Arora, Sabri Eyuboglu, Aman Timalsina, Isys Johnson, Michael Poli, James Zou, Atri Rudra, Christopher Ré

Figure 1 for Zoology: Measuring and Improving Recall in Efficient Language Models
Figure 2 for Zoology: Measuring and Improving Recall in Efficient Language Models
Figure 3 for Zoology: Measuring and Improving Recall in Efficient Language Models
Figure 4 for Zoology: Measuring and Improving Recall in Efficient Language Models
Viaarxiv icon

FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

Add code
Bookmark button
Alert button
Nov 10, 2023
Daniel Y. Fu, Hermann Kumbong, Eric Nguyen, Christopher Ré

Viaarxiv icon

Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture

Add code
Bookmark button
Alert button
Oct 18, 2023
Daniel Y. Fu, Simran Arora, Jessica Grogan, Isys Johnson, Sabri Eyuboglu, Armin W. Thomas, Benjamin Spector, Michael Poli, Atri Rudra, Christopher Ré

Viaarxiv icon