Alert button

GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

Add code
Bookmark button
Alert button
May 22, 2023
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai

Figure 1 for GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Figure 2 for GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Figure 3 for GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Figure 4 for GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

Share this with someone who'll enjoy it:

View paper onarxiv icon

Share this with someone who'll enjoy it: