Picture for Hermann Kumbong

Hermann Kumbong

The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

Feb 06, 2024
Figure 1 for The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
Figure 2 for The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
Figure 3 for The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
Figure 4 for The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
Viaarxiv icon

FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

Add code
Nov 10, 2023
Figure 1 for FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
Figure 2 for FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
Figure 3 for FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
Figure 4 for FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
Viaarxiv icon

Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions

Oct 28, 2023
Figure 1 for Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Figure 2 for Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Figure 3 for Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Figure 4 for Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Viaarxiv icon