Alert button

Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

Aug 27, 2021
Ofir Press, Noah A. Smith, Mike Lewis

Figure 1 for Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Figure 2 for Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Figure 3 for Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
Figure 4 for Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

Share this with someone who'll enjoy it:

View paper onarxiv icon

Share this with someone who'll enjoy it: