Picture for Xunhao Lai

Xunhao Lai

Model Merging in Pre-training of Large Language Models

Add code
May 17, 2025
Viaarxiv icon

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference

Add code
Feb 28, 2025
Viaarxiv icon

Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?

Add code
Oct 02, 2024
Viaarxiv icon