Picture for Damai Dai

Damai Dai

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models

Add code
Jul 02, 2024
Viaarxiv icon

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

Add code
Jun 17, 2024
Viaarxiv icon

Exploring Activation Patterns of Parameters in Language Models

Add code
May 28, 2024
Viaarxiv icon

Large Language Models Are Unconscious of Unreasonability in Math Problems

Add code
Mar 28, 2024
Figure 1 for Large Language Models Are Unconscious of Unreasonability in Math Problems
Figure 2 for Large Language Models Are Unconscious of Unreasonability in Math Problems
Figure 3 for Large Language Models Are Unconscious of Unreasonability in Math Problems
Figure 4 for Large Language Models Are Unconscious of Unreasonability in Math Problems
Viaarxiv icon

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization

Add code
Feb 25, 2024
Figure 1 for PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
Figure 2 for PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
Figure 3 for PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
Figure 4 for PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
Viaarxiv icon

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Add code
Jan 11, 2024
Viaarxiv icon

Language Models Understand Numbers, at Least Partially

Add code
Jan 08, 2024
Viaarxiv icon

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

Add code
Jan 05, 2024
Figure 1 for DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
Figure 2 for DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
Figure 3 for DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
Figure 4 for DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
Viaarxiv icon

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations

Add code
Dec 28, 2023
Viaarxiv icon

Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning

Add code
Oct 12, 2023
Figure 1 for Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning
Figure 2 for Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning
Figure 3 for Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning
Figure 4 for Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning
Viaarxiv icon