Alert button
Picture for Wenxiang Hu

Wenxiang Hu

Alert button

DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM

Oct 23, 2023
Weijie Xu, Wenxiang Hu, Fanyou Wu, Srinivasan Sengamedu

Figure 1 for DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM
Figure 2 for DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM
Figure 3 for DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM
Figure 4 for DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM

In the burgeoning field of natural language processing, Neural Topic Models (NTMs) and Large Language Models (LLMs) have emerged as areas of significant research interest. Despite this, NTMs primarily utilize contextual embeddings from LLMs, which are not optimal for clustering or capable for topic generation. Our study addresses this gap by introducing a novel framework named Diffusion-Enhanced Topic Modeling using Encoder-Decoder-based LLMs (DeTiME). DeTiME leverages ncoder-Decoder-based LLMs to produce highly clusterable embeddings that could generate topics that exhibit both superior clusterability and enhanced semantic coherence compared to existing methods. Additionally, by exploiting the power of diffusion, our framework also provides the capability to generate content relevant to the identified topics. This dual functionality allows users to efficiently produce highly clustered topics and related content simultaneously. DeTiME's potential extends to generating clustered embeddings as well. Notably, our proposed framework proves to be efficient to train and exhibits high adaptability, demonstrating its potential for a wide array of applications.

* EMNLP 2023  
* 19 pages, 4 figures, EMNLP 2023 
Viaarxiv icon

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Jun 14, 2023
Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, Daxin Jiang

Figure 1 for WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Figure 2 for WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Figure 3 for WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Figure 4 for WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM

* Large Language model, Code Generation, Code LLMs 
Viaarxiv icon